October)

manager Editor: Donald J. Reifer

■

Reifer Consultants

■

[email protected]

Eight Secrets of Software Measurement Betsy Clark

12

IEEE SOFTWARE

wenty years ago, I left the security (and frustration) of working for a large corporation to begin a consulting career in software measurement. Since then, I’ve helped many firms implement software measurement programs. For some clients, the motivation for measurement was process improvement; for others, it was resolving an immediate crisis and getting a product out the door. In this column, I’ll share the eight “secrets” of software measurement that I’ve learned. I call them secrets because they were not obvious to me at the beginning. Only in retrospect, as I’ve tried to discern patterns of success and failure, have these secrets become clear.

T

To create an effective measurement program, you first must understand exactly where you want to go or what you want to accomplish; that is the “why” of measurement.

It’s not about the metrics Four-time Tour de France winner Lance Armstrong titled his autobiography It’s Not About the Bike: My Journey Back to Life (Berkeley Publishing Group, New York, 2001). Although he has spent countless hours on the bike, for him, it was only a vehicle for his fight back from life-threatening cancer. By the same token, measurement is not an end in itself; it’s a vehicle for highlighting activities and products that you, your project team, and your organization value so you can reach your goals. But it’s only a tool. To get anywhere, you must navigate the road— you’ve got to make decisions and act.

Establishing a measurement program is easy; keeping it going is hard I am continually impressed by how easy it is to think about potentially useful measures and how hard it is to implement an effective measurement program. Within a project or organization, it’s often easy to get people enthused about measures—but all too often, that enthusiasm does not translate into action. Even when it does, it is unlikely to be sustained. Getting the numbers is easy; doing something with them is not. What you need is no less than to change your organization’s culture. Cultural change is hard.

September/October 2002

Success comes from channeling an organization’s pain into action No matter how much I dislike this secret, I have found it to be so. It comes back to the fact that it’s not about the metrics; it’s about the strength of the motivation to know or improve something and to follow through with action. No matter how noble the intention, “Let’s do metrics” just doesn’t provide sufficient motivation. The single biggest determinant of measurement success lies in the answers to the following questions: How badly do you want to know the information, and how will you use it?

0740-7459/02/$17.00 © 2002 IEEE

MANAGER

People skills matter more than quantitative skills Every step of the measurement process requires input from the people within the project or organization who will provide and use the data. Emotion plays a strong role, especially at the beginning, leading to a variety of reactions from the individuals involved. Fear that the measures will be misinterpreted or misused can also lead to resistance. (We’ll address this in more detail below.) Positive and negative reactions accompany any organizational change; anticipating and managing these are necessary. In my experience, at least one person always steps up as an early adopter. This person intuitively understands the need for measurement and its benefits. I make sure to find that person and work with him or her. By demonstrating that providing the data isn’t so hard and that it really is useful, such people can bring credibility to the measurement activities. By the same token, I almost always encounter someone who feels threatened by the measurement program. This person—usually a longterm middle manager—often derives a sense of importance as the repository of organizational knowledge. Measurement sheds light on the organization’s basic workings; so much of the knowledge previously held in that person’s head now becomes an organizational asset accessible by all. That’s good news for the organization but can be threatening to the particular individual who might lose status as the company’s resident sage. Senior-level sponsorship and leadership are critical Although this one might be so obvious that it really shouldn’t qualify as a secret, it’s absolutely key and often neglected. Remember that it’s not about the metrics; it is about ar-

ticulating a vision and following through with consistent and persistent action. Without these, measurement won’t help. The person at the top must participate in the measurement program, by ■ ■ ■ ■ ■ ■

Articulating organizational goals Behaving in ways that are consistent with these goals Creating a culture that exposes, rather than hides, problems Looking at the data and asking questions Making decisions and following through with action Expecting lower-level managers to do the same

Effective measurement will expose an organization’s warts. To improve a situation, you must first understand where you are. In some ways, things might seem to get worse before they get better. They aren’t really worse, but you are now, perhaps for the first time, bringing problems to light. Dealing with this challenge effectively takes courage and fortitude.

Measuring individuals can be okay This is probably the most controversial secret. Every source of guidance I’ve read on measurement advises against measuring individuals. In conference presentations and casual conversation, people often repeat this observation. In my view, there are counterproductive ways of measuring individuals, but there are also times when it is appropriate. Let’s look first at some counterproductive ways. Punishing people for reporting honest results can quickly destroy data integrity. Organizations with effective measurement programs do punish people who hide risks and problems; they don’t punish people for exposing them, especially when they also offer solutions. Rewarding one programmer for producing more lines of code per labor hour or punishing someone else for producing less is also inappropriate. That course ignores the fact that code quality and inherent task difficulty vary. In looking at intelligent uses, I’ve worked with clients who have implemented detailed progress measures that begin with the individual, measuring actual progress against a plan. The measures can be rolled up, leading to an overall progress assessment, and they can be drilled down, to the level of teams or individuals, supporting a detailed assessment of problem areas. Such an approach lets managers hold individuals accountable for completing assignments as planned. There’s nothing negative about this. Don’t go overboard trying to be perfect Anyone who has gotten down and dirty with data quickly realizes the importance of understanding what the data represent. Even seemingly straightforward data can be ambiguSeptember/October 2002

IEEE SOFTWARE

13

MANAGER

ous. One of my early consulting engagements in capturing milestone dates brought this point home to me. When I asked for the date of a design review, I was asked, in turn, “Are you asking for the date the review was held or the date the customer signed off on it nine months later?” It is very important to understand what the data represent. By the same token, keep in mind that “the good is the enemy of the perfect.” You must continually balance clarity of definition with the need to get started. In my view, it’s best to get started and work to improve the measurement process over time. Understanding the reasons for variability in the data provides a powerful decision tool One of the striking characteristics of real data is its huge variation. Look at any graph showing the relationship between size and effort or

between size and defects and you’ll see a very large spread—so large that these data are typically represented in logarithmic scales. If you can understand what’s behind the variation, you’ll understand a lot. I once had a client who was embarking on a process improvement initiative. They wanted to baseline where they were at the beginning to establish a comparison point for assessing the impact of their improvement activities over time. In typical fashion, the relationship between size and effort varied across projects by an order of magnitude. They measured a number of characteristics about each project, including the project’s percentage of personnel turnover. In this organization, turnover had a major impact on productivity. When turnover increased from 12 percent to 24 percent per year, the associated effort increased by 36 percent. In dollar terms, if a project with low turnover costs US$250,000, then one with high turnover would cost

US$340,000. Within this organization, it makes monetary sense to minimize turnover.

I

mplementing an effective measurement program is full of challenges. Overcoming these challenges is worthwhile because measures provide insight supported by hard data. Measurement provides a vehicle for improving your ability to plan and track progress and for addressing risks and problems earlier. You’ll know where you are and where you’re going. You still have to set the direction and do the steering, but measurement wil be an important tool in navigating the road to success.

Betsy Clark is President of Software Metrics, a Virginia-

based measurement consulting company she co-founded in 1983. She received her BA from Stanford University and PhD from the University of California, Berkeley, both in cognitive psychology. Contact her at [email protected].

NEW FOR 2002 the IEEE Computer & Communications Societies present

IEEE PERVASIVE COMPUTING The exploding popularity of mobile Internet access, third-generation wireless communication, and wearable and handheld devices have made pervasive computing a reality. New mobile computing architectures, algorithms, environments, support services, hardware, and applications are coming online faster than ever. To help you keep pace, the IEEE Computer Society and IEEE Communications Society are proud to announce IEEE Pervasive Computing. This new quarterly magazine aims to advance mobile and ubiquitous computing by bringing together its various disciplines, including peer-reviewed articles on

• • • • •

Hardware technologies Software infrastructure Real-world sensing and interaction Human–computer interaction Security, scalability, and privacy

Editor in Chief M. Satyanarayanan Carnegie Mellon Univ. and Intel Research Pittsburgh Associate EICs Roy Want, Intel Research; Tim Kindberg, HP Labs; Deborah Estrin, UCLA; Gregory Abowd, GeorgiaTech.; Nigel Davies, Lancaster University and Arizona University

SUBSCRIBE NOW! http://computer.org/pervasive

requirements Editor: Suzanne Robertson

■

The Atlantic Systems Guild

■

[email protected]

Requirements and Testing: Seven Missing-Link Myths Dorothy Graham

Testing expert Dorothy Graham asserts that we can save a great deal of time and money if testers are involved in testing requirements. If the requirements have some consistent quality criteria, testers can raise questions and find problems before we turn them into code. —Suzanne Robertson

A

strong link between testing and requirements engineering can benefit both sides, but often this link is missing. Let’s examine the seven most common myths or misconceptions behind this missing link.

Myth 1: Requirements at the beginning, testing at the end “We don’t need to think about testing yet. Let’s just concentrate on requirements.” This attitude guarantees that you will have a rough time at the end of your project. Of course, getting good requirements is important, but getting testers involved during requirements analysis is one of the best ways to ensure good requirements. As Figure 1 shows, you should perform test design activities as soon as there is something to design tests against—usually during the requirements analysis for system and user acceptance tests. A common problem, especially for testers, is the impact of late changes on requirements. Suppose you’re in the last weeks of system testing, and user acceptance tests are scheduled to start running in two weeks. Suddenly, your users say, “By the way, we’d like the system to do this differently,” which is extremely frustrating and happens more often than it should. But, what are the users really doing? 0740-7459/02/$17.00 © 2002 IEEE

They are designing their acceptance tests. The act of test design highlights what they really want the system to do. If they had designed their tests early (left side of Figure 1), they could have discovered these problems before they were built into the system. Getting users involved in both requirements and testing is crucial. Have you ever bought a used car? Would you go to the salesperson and say, “You know more about cars than I do, so take the test drive for me”? If you do, you deserve what you get. Similarly, users often say to techies, “You know more about computers than we do, so do the acceptance testing for us.” Caveat emptor! Myth 2: Testing isn’t possible until the system exists “We can’t do any testing because nothing has been built yet. Testers just play with the system and see what happens. Anyway, you can’t test a piece of paper.” Three things are wrong with this sentiment. First, testing is more than just seeing what happens. It’s far more rigorous and systematic than that. Second, it’s more than just executing tests. As Figure 2 shows, executing tests and checking results is part of the fundamental test process, but other important activities exist as well. Third, you can and should test written requirements documents against business or project objectives for completeness and correctness. If you don’t test requirements on paper, you September/October 2002

IEEE SOFTWARE

15

REQUIREMENTS DEPT TITLE

quirements analysis benefit from having this feedback loop in place. Good requirements engineering produces better tests; good test analysis produces better requirements.

Test des Business requirements

Acceptance testing Test des

Project specification

System integration testing Test des

System specification

System testing Test des

Design specification

Component integration testing Test des Component testing

Code Design tests

Figure 1. A V-model for early test design.

Run tests

will build errors into the system that you could have easily removed much earlier. Myth 3: Requirements are used in testing, but not vice versa “You don’t test requirements—you test from them.” A tester’s mindset differs from a developer’s. It’s fairly easy to write a requirement that’s a bit vague and ambiguous but appears to be OK. However, when good testers look at a requirements specification, they devise specific test cases to flush out vague, ambiguous, or unclear requirements. When you try to explain an abstract concept to someone, you say “for example” and illustrate the idea with concrete and specific cases to clarify the concept. Tests are the “for examples” for requirements. Think of “what if” use cases or business scenarios. If you consider how a particular user will use the system, the system functionality that seemed abstract when first described becomes specific for that particular user. Both testing and re-

Policy and strategies Test planning (at each level) Test Design test cases process Build tests Execute (run) tests Check results Check exit criteria, test report

Figure 2. What is “testing”? 16

IEEE SOFTWARE

September/October 2002

Process improvement

Identify conditions

Myth 4: If writing tests is difficult, it’s solely a test problem “The testers seem to have problems writing tests from our requirements—maybe we should get some better testers.” Not all requirements are created equal from a tester’s perspective. Specifying tests for some of them is easy; for others, identifying exactly what the system should do (and thus identifying tests to verify that it can do these things) is a nightmare. Specifying testable nonfunctional requirements such as usability or performance is difficult.1 Phrases such as easy to use, user friendly, very reliable, or acceptable performance are not specifications: they are vague, ethereal intentions. I subscribe to Tom Gilb’s law: “Anything can be made measurable in a way that is superior to not measuring it at all.”2 Note that it is not made measurable in a perfect way but a useful way. Suzanne and James Robertson’s “fit criteria” show specifically how to make requirements measurable and therefore testable.3 Myth 5: Minor changes in requirements won’t affect the project (much) “Just add a couple more spaces to this input field. There’s plenty of room on the screen. It’s a minor change; you won’t need to test it because it’s so small.” A change that appears to be minor from a requirements viewpoint could have a far-reaching impact, especially in testing. Suppose that adding two more characters to a field means that the database where this field is defined must be rearranged. What if this field corresponds to similar information held elsewhere in the system? What if the routines that check this field also check other fields that you have not increased in length? Do you now need two checking routines? You must test all such changes to confirm that the system is now doing the right thing in every situation affected by this change. In addition, some unexpected side effects could arise, so you should also do regression testing to ensure that the system has not regressed in any other areas. How much testing you do depends on the risks of a change having both known and unknown impacts on the system. Testing can also help

REQUIREMENTS DEPT TITLE

mitigate the risk of change by giving confidence that such impacts are low. Myth 6: Testers don’t really need requirements “I know we don’t have good requirements for this system, but test it as best you can—just see what the system does.” A tester’s job is to adjudicate between what a system does and what it should do. The system should help the business accomplish a goal, so what the system actually does should be compared with those goals. There is an oracle assumption in testing (which has nothing to do with databases or the companies that supply them—rather, it is based on the oracle of Delphi, who could predict the future with unerring accuracy). The oracle assumption states that the tester routinely knows what the correct answer is supposed to be, which is fundamental to testing. A test comprises the test inputs, preconditions, and expected outcomes. How do you know what the expected outcome should be? A requirement specification is this test basis, or oracle. So, yes, testers do need requirements; otherwise, you could argue that it’s not really a test. Myth 7: Testers can’t test without requirements “Because requirements form the basis for tests, obviously we can’t do any testing until we have decent requirements.” This is also a common tester’s misconception. Sometimes changes are made to systems where requirements are inadequate or nonexistent. This makes the testing more difficult, but you shouldn’t just throw your hands up and say that you can’t do it. Without requirements, testers still need some kind of test oracle—maybe users who are familiar with the way the system works, the old system, user manuals, or the tester’s own opinions. What happens if the test oracle is the tester? Instead of merely comparing a system against a document, the testers are judging the system against their personal view of what it should do. (Actually, testers should always do some of this anyway, but that’s another story.) Some would say that without a specification, you are not testing but merely exploring the system. This is formalized in the approach known as exploratory testing, designed for situations with inadequate requirements and severe time pressure.4 If you end up with a good set of testware,

test plans, test specifications, and so on, the testware actually becomes the only requirement documentation in the project. So with good enough tests, who needs requirements at all? Do the testers become the requirements engineers? Is this a good idea? If requirements are poor or nonexistent, testers must do the best testing they can under less than ideal circumstances. Testing is far more rigorous if it is based on good requirements, and tester involvement early on can help produce them.

I

hope I’ve convinced you that testers have much to offer in producing better requirements. Here’s how to achieve it in practice:

■

Invite testers to participate in requirement reviews and inspections Begin planning testing in parallel with requirements analysis Ask for sample test conditions and test cases to use as examples in the requirements specification Include in the requirements document any specific cases that you had in mind when analyzing requirements Use business scenarios and use cases to give specific examples of how the system should work Set measurable criteria for both functional and nonfunctional requirements

■ ■

■

■

■

Testing is challenging in two ways: it’s a rewarding intellectual activity, but it also challenges whatever the tests are based on. Make the link between requirements and testing. If you accept and encourage the challenges that testers make to your requirements, you will avoid these misconceptions and will end up with significantly better requirements and tests. Acknowledgments Thanks for comments on drafts of this article from Clive Bates, Mark Fewster, Robin Goldsmith, and Dot Tudor.

References 1. B. Lawrence, K. Weigers, and C. Ebert, “The Top Ten Risks of Requirements Engineering,” IEEE Software, vol. 18, no. 6, Nov./Dec. 2001, pp. 62–63. 2. T. Gilb, Principles of Software Engineering Management, Addison Wesley, Boston, 1988. 3. S. Robertson and J. Robertson, Mastering the Requirements Process, Addison Wesley, Boston, 1999. 4. C. Kaner, J. Bach, and B. Pettichord, Lessons Learned in Software Testing, John Wiley & Sons, New York, 2002.

Good requirements engineering produces better tests; good test analysis produces better requirements.

Dorothy Graham founded Grove Consultants in the UK, a company that provides consultancy, training, and inspiration in all aspects of software testing. Contact her at [email protected]. September/October 2002

IEEE SOFTWARE

17

design Editor: Martin Fowler

■

T h o u g h t Wo r k s

■

[email protected]

How .NET’s Custom Attributes Affect Design James Newkirk and Alexei A. Vorontsov

I

n its first release of the .NET Framework, Microsoft has provided a defined method for adding declarative information (metadata) to runtime entities in the platform. These entities include classes, methods, properties, and instance or class variables. Using .NET, you can also add declarative information to the assembly, which is a unit of deployment that is conceptually similar to a .dll or .exe file. An assembly includes attributes that describe its identity (name, version, and culture), informational attributes that provide additional product or company information, manifest attributes that describe configuration information, and strong name attributes that describe whether the assembly is signed using public key encryption. The program can retrieve this metadata at runtime to control how the program interacts with services such as serialization and security. Here, we compare design decisions made using custom attributes in .NET with the Java platform. Marker interfaces In the Java platform there is a common design trick called marker interfaces. A marker interface has no methods or fields and serves only to identify to the Java Virtual Machine (JVM) a particular class attribute. Here is one example: public interface Serializable {}

If the class that you are writing must be se18

IEEE SOFTWARE

September/October 2002

rializable, then it must implement the interface as follows: public class Serializable

MyClass

implements

Serialization might have certain behaviors associated with it that the class developer wants to control. However, Java doesn’t explicitly associate such behavior with the interface that represents the serialization contract. At runtime, when the program tells the JVM to serialize this class, it looks to see if the class has implemented the interface. It also looks to see if the class has defined any special methods associated with the serializable interface but not directly declared in it, such as readResolve, readObject, or writeObject. The JVM relies on a naming convention and method signatures to locate the methods via reflection; if it finds them, it invokes them. The interface itself does not specify any methods, because implementors might then unnecessarily implement methods in the simplest case. Because the interface doesn’t explicitly specify the methods used to control the process and thus might incorrectly specify the method signature, this mechanism is prone to failure. Unfortunately, no compile time check will identify this as an error. .NET solves this problem by being explicit. In the simplest case, where the programmer wants to rely on the provided capability to serialize an object, there is a class-level attribute called Serializable 0740-7459/02/$17.00 © 2002 IEEE

DESIGN

that marks a class as having that capability. For example, [Serializable()] public class MyClass {}

Marking a class serializable implies nothing else. If the programmer wants to completely control the serialization process, then he or she must also implement an interface called ISerializable, specifying the methods used to control serialization (see Figure 1). At runtime, when the program tells the Common Language Runtime to serialize a class, the CLR looks to see if the class was marked with SerializableAttribute. The Java and .NET approaches are similar in intent, but .NET’s use of attributes is more explicit. Contrary to an interface’s basic purpose, the marker interface reuses an existing language construct interface to represent what the attribute represents. According to the Java Language Specification (2nd ed., Addison-Wesley, 2000), An interface declaration introduces a new reference type whose members are classes, interfaces, constants and abstract methods. This type has no implementation, but otherwise unrelated classes can implement it by providing implementations for its abstract methods. Stylistic naming patterns At the method level, it is common in the Java platform to use naming conventions to identify a special method. By virtue of the name, the program finds the method at runtime using reflection. Once found, the executing program specially interprets this method. For example, when a programmer defines a test method in JUnit—a popular unittesting framework for Java (www. junit.org)—the first four letters of a test method must be test (see Figure 2a). The program that executes the tests first verifies that the class inherits from TestCase. Then, using reflection, it looks for any methods that start with Test. In the code in Figure 2a, the pro-

gram finds and calls testSuccess.

[Serializable()]

public class MyClass : ISerializable The code in Figure { 2b demonstrates a public MyClass( common design idSerializationInfo info, iom used in JUnit StreamingContext context) when the program{ mer wants to verify // ... that the code throws } an exception. Unforpublic void GetObjectData( tunately, the proSerializationInfo info, grammer will dupliStreamingContext context) cate such code in { every test case that // ... expects an exception, } and the idiom is not } as intuitive as you might expect. Having the testing framework support Figure 1. Implementing the ISerializable interface, which specifies the methods for such a common case controlling serialization. directly would be nice. However, relying on the naming convention could rivative of JUnit, supports all lanlead to some variation of the code in guages in the .NET framework and Figure 2c. In this case, we use a nam- uses attributes at the class and ing convention to specify not only a method levels. The class attribute is test method but also additional infor- called TestFixture; it tells the promation about how to interpret the gram that runs the tests to look for test result. We expect that this test methods in this class. A Test atmethod’s execution will throw MyEx- tribute then identifies test methods. ception. This example might seem This overall solution makes for a somewhat ridiculous, but it demon- more consistent approach. strates the limitations of naming conIn addition, this solution is more ventions, because of how much infor- extensible because more than one atmation the name itself can convey. In tribute can be associated with a fact, JUnit doesn’t implement the method, and attributes can have adfunctionality to check boundry condi- ditional data. For example, Nunit tions in this way. Other approaches has another attribute defined for a used in Java (such as JavaDoc tags) method that expects an exception. can provide additional information. This leaves the name not only unenHowever, they are not present at run- cumbered by the context in which it time and usually require preprocess- is run but also more relevant to what ing the code to identify and process is being tested (see Figure 3b). the tags. In .NET, stylistic naming patterns are not needed because, in addition to attributes that the Framework ttributes in .NET provide an elespecifies, a programmer can create gant, consistent approach to custom attributes that are defined adding declarative information to and used in the same way. These at- runtime entities. Because the runtime tributes are not just names but are entities interact with the supporting instances of classes that might have services via declarative information, additional data. Figure 3a shows a the set of services and supporting atsimilar test class defined with Nunit tributes does not have to be closed. (www.nunit.org), a unit testing tool By providing a standard mechanism for the .NET platform. Nunit, a de- to extend built-in metadata with cus-

A

September/October 2002

IEEE SOFTWARE

19

DESIGN

public class MyClass extends TestCase { public void testSuccess() { /* ... */ } } (a) public class MyClass extends TestCase { public void testMyException() { try { /* code that throws exception */ fail(“Code should have thrown MyException”); } catch(MyException e) { /* expected exception — success */ } } } (b) public class MyClass extends TestCase { public void testSuccess_ExpectException_MyException() { /* ... */ } } (c)

tom attributes, .NET lets programmers develop applications that can interact with services not yet defined or supported by CLR. In fact, Nunit version 2.0 was written with custom attributes and provides much of the flexibility we’ve demonstrated here. In contrast, the most common ad hoc mechanisms in Java to add declarative information include marker interfaces, stylistic naming patterns, and JavaDoc tags. These inconsistently solve the problem of adding declarative information to runtime entities. They are also error prone and too simplistic for today’s applications. The Java community recognizes this limitation and has started working on JSR-175 (see www.jcp. org/jsr/detail/175.jsp), which specifies a similar facility for Java that is already in .NET. James Newkirk is a software project manager for

Thoughtworks. He has been working with the .NET Framework since its introduction in the summer of 2000. Contact him at [email protected].

Alexei A. Vorontsov is a software technical lead for

Figure 2. (a) A test method in JUnit (the method’s first four letters must be test); (b) a test for the boundary conditions that verify that an exception is thrown when expected; (c) a naming convention to specify not only a test method but also some additional information about how to interpret the test result.

[TestFixture] public class MyClass { [Test] public void Success() { /* ... */ } } (a) [TestFixture] public class MyClass { [Test] [ExpectedException(typeof(MyException))] public void Success() { /* would throw my exception */ }

Thoughtworks. He has worked on an enterprise transforming application for the past three years. Contact him at alexei@ nunit.org.

IEEE

FUTURE TOPICS: The Business of Software Engineering

Visit us onInspections the Software Usability web Internationalization

} (b)

Figure 3. A test class (a) defined with Nunit and (b) with another attribute defined for a method that expects an exception. 20

IEEE SOFTWARE

September/October 2002

http://computer.org/software

memorial Senior Lead Editor: Dale Strok

■

s o f t w a r e @ c o m p u t e r. o r g

From Goto-less to Structured Programming:

The Legacy of Edsger W. Dijkstra Peter P. Chen

D

id you know that Edsger Dijkstra just passed away?” When I first heard the news, I was shocked. “How could this happen?!” I asked myself. Last year (but it seemed just yesterday) when Edsger and I sat together on a small bus leaving the Software Pioneers Conference in downtown Bonn, Germany, to return to our hilltop hotel, he looked healthy. On that bus ride, we discussed the interesting difficulties he had faced early on in his career. Although he had run into obstacles, he could now laugh about them since he had discovered ways to overcome them and change history. Today as I recall his face and soft voice, I see the smile of a man who revolutionized software development. A short biography Edsger W. Dijkstra was born in the Netherlands in 1930 and died 6 August 2002. After receiving his PhD in computing science from the University of Amsterdam, he worked as a programmer at the Mathematical Centre in Amsterdam, a math professor at Eindhoven University of Technology, a research fellow at the Burroughs Corp., and the Schlumberger Centennial Professor of Computer Science at the University of Texas at Austin. He received the ACM Turing Award in 1972. 0740-7459/02/$17.00 © 2002 IEEE

I am a “programmer!” The first time I saw Dijkstra was when he delivered his Turing Award acceptance speech. I was one of the many people who stood for an hour to listen to that speech because all the seats were filled. It was worth it: he told us a story I will never forget. When he applied for a wedding license in 1957, Dutch law required him to declare his profession. Filling in the form, Dijkstra stated he was a “programmer.” The Amsterdam authorities claimed there was no such profession and rejected his initial application. As a result, his marriage certificate stated his profession as “theoretical physicist.” What struck me 30 years ago and still resonates in my mind today is how Dijkstra was proud to be a programmer instead of a theoretical physicist. This is the kind of person software development needs; being proud of one’s profession is one of the most crucial psychological steps toward betterquality work. Major contributions Dijkstra’s most famous paper is probably “Goto Statement Considered Harmful” (Comm. ACM, Mar. 1968, pp. 147–148), which brought considerable attention to the problem of software developers’ careless usage of the Goto statement. As a result, programmers today use it more carefully or not at all.

In 1972, Dijkstra published “Notes on Structured Programming” (Structured Programming, O.J. Dahl, E.W. Dijkstra, and C.A. Hoare, eds., Academic Press, 1972). This triggered the Structured Programming movement, which helped many of us improve our practices. A survey of more than 1,000 college professors identified the 38 most influential papers in computer science (Great Papers in Computer Science, P. Laplante, ed., West Publishing Co., 1996; www.csc.lsu.edu/~chen/greatpapers. htm), and Dijkstra authored five of them. In June 2001, at the Software Pioneers Conference, about 1,200 software professionals saw Dijkstra speak for the last time. Fortunately, that speech is preserved in streaming video (www.sdm.de/conf2001/index_ e.htm) and book/DVD (Software Pioneers: Contributions to Software Engineering, M. Broy and E. Denert, eds., Springer-Verlag, 2002) format.

E

dsger Dijkstra is one of the most influential figures in computer science. His teachings (www.cs.utexas. edu/users/EWD) will resonate through the work of software developers for many years to come.

Peter P. Chen is the Foster Distinguished Professor at Louisiana State University and the inventor of the EntityRelationship model. Contact him at [email protected].

September/October 2002

IEEE SOFTWARE

21

focus

guest editors’ introduction

The Impending Changes in Software Education Thomas B. Hilburn, Embry Riddle Aeronautical University Watts S. Humphrey, The Software Engineering Institute, Carnegie Mellon University

ecause of the growing impact of software and its historically poor performance in meeting society’s needs, the practice of software engineering is in need of substantial changes. One challenge concerns preparing software professionals for their careers; the field must drastically change its approach to software engineering education if it hopes to consistently provide safe, secure, and reliable systems. This

B

special issue sheds light on some of the issues both academic and industry-based educators must resolve to better address society’s need for qualified and capable software professionals. Software’s critical nature Software is increasingly used in businesscritical and even life-critical situations. We are moving from a benign environment with typically friendly and honest users to an environment in which criminals, malcontents, and even terrorists use our systems. Safety and security are growing concerns. However, although the public’s health, safety, and welfare increasingly depend on software, approximately 75 percent of all software projects are either late or cancelled.1 A staggering 78 percent of IT organizations have been involved in disputes 22

IEEE SOFTWARE

September/October 2002

0740-7459/02/$17.00 © 2002 IEEE

ending in litigation, and 67 percent of those cases involved late delivery or total project failure.2 In nearly half the cases, the software was so defective it was unusable. Just in the US manufacturing and transportation industries alone, inadequate software infrastructures cost $5.85 billion annually.3 Political action Society’s typical reaction to industrial problems affecting the public is instructive. The government typically regulates industries concerning public health, safety, and security—for example, nuclear power, medical devices, food, drugs, and aviation all require regulation. When the public feels an industry isn’t sufficiently protecting its interests, problems cease to be technical and become political. When faced with a public outcry, politicians often enact laws and set up regulatory bodies so organizations cannot release products or provide services without governmental approval. These fears might seem unrealistic and overstated, but consider the history of political action when faced with repeated industrial failure. If the software industry can’t solve its own problems, the government might establish an oversight group to protect the public. All it would take is a highly visible disaster to galvanize public opinion and motivate political action. Although the pace of political action is typically glacial, the political process has already started. States are now considering the licensing of software professionals; Texas has already enacted such legislation, and two state legislatures have enacted the controversial Uniform Computer Information Transactions Act (UCITA) proposal to limit industrial liability for software failures. (The National Conference of Commissioners on Uniform State Laws developed the UCITA proposal to update the Universal Commercial Code for the information age. It provides a legal framework for interpreting contracts involving software and other intellectual property. Industry groups are now trying to get all 50 US states to enact UCITA legislation.) The software problem is now a political concern, and we can only hope we find sound technical solutions before the public demands governmental protection from our inadequate performance.

Can we solve our own problems? Can the software industry solve its own problems in time to prevent a serious public disaster, or must the government solve our problems for us? The fact that a governmental body doesn’t know how to solve these problems any better than we do will not deter politicians. If the problems are serious enough, and the public concerned enough, there will be action—whether it’s effective or not. Because today’s software professionals can’t consistently and predictably produce quality software, we must change how we develop and support software and how we educate, train, and manage software professionals. Because a few leading organizations can consistently produce quality software, this must be at least partly an education problem. A fundamental question concerns the mix of theory and practice in software engineering education. In this issue’s Point/Counterpoint department, Barry Boehm compares software to a contact sport, claiming software professionals must be prepared for the industry’s rough-and-tumble life. He argues for an increased focus on economics, change management, and practical experience. Conversely, Allen Tucker argues that software professionals need a stronger grounding in theory and disciplined practice to consistently produce quality software. The theory-versus-practice debate raises the issue of how completely the academic community can and should prepare graduates for an industrial software career. For example, is software like mechanical and electrical engineering, where new graduates must know the basics to get their first job, and then they can start their lifelong career of learning how to become professional engineers? Or should software education more closely mirror the medical profession, with its intense specializations, teaching hospitals, internships, and licensing? Two of the articles in this issue, one by Richard Conn and the other by Jorge Diaz-Herrera, Mike Murphy, and Dawn Ramsey, discuss industry needs, focusing on the quality and commitment problems industry faces and how better preparation could help address these problems. Dale Callahan and Bob Pedigo address the IT community’s needs, including graduate programs for IT managers. September/October 2002

IEEE SOFTWARE

23

Suggested Reading D. Bagert et. al., Guidelines for Software Engineering Education, Version 1.0, tech. report CMU/SEI-99-TR-032, Software Engineering Institute, Carnegie Mellon Univ., 1999. N. Coulter and N. Gibbs, eds., Annals of Software Engineering, vol. 6, Apr. 1999. P.J. Denning, “Educating a New Engineer,” Comm. ACM, vol. 35, no. 12, Dec. 1992, pp. 83–97. G. Ford and N. Gibbs, A Mature Profession of Software Engineering, tech. report, CMU/SEI-96-TR-004, Software Engineering Institute, Carnegie Mellon Univ., 1996, www.sei.cmu.edu/publications/ documents/96.reports/96.tr.004.html, 1996. W.S. Humphrey, Introduction to the Personal Software Process, Addison-Wesley, Boston, 1997. W.S. Humphrey, The Discipline of Software Engineering, AddisonWesley, Boston, 1995. W.S. Humphrey, Introduction to the Team Software Process, AddisonWesley, Boston, 2000. W.S. Humphrey, Winning With Software: An Executive Strategy, Addison-Wesley, Boston, 2002. T. Lethbridge, “What Knowledge is Important to a Software Professional,” Computer, vol. 33, no. 5, May 2000, pp. 44–50. Joint IEEE-CS/ACM Task Force on Computing Curriculum, Computing Curriculum 2001, Volume II, Dec. 2001; www.computer.org/education/ cc2001/final/index.htm.

Four articles discuss the tradeoffs involved in designing software curricula. Hossein Saiedian, Donald Bagert, and Nancy Mead address the academic considerations involved in developing software engineering programs. Jürgen Börstler, David Carrington, Gregory Hislop, Susan Lisack, Keith Olson, and Laurie Williams describe the curriculum issues involved in introducing disciplined software engineering methods and in structuring and teaching a Personal Software Process course. Ken Surendran, Helen Hays, and Andrew Macfarlane describe how apprenticeship programs provide practical exposure to industrial problems and issues, and Grant Cheston and Jean-Paul Tremblay discuss introducing software engineering courses at the beginning of an academic program. The final three articles deal with team project courses, discussing the skills involved and disciplines required to work on software teams. The trade-offs concern the rigor required to produce quality work and students’ willingness to enroll in rigorous academic programs. The steady decline in computer science and engineering enrollments suggests that these more rigorous methods must be justified or student enrollments will continue to drop. In our article, 24

IEEE SOFTWARE

September/October 2002

we discuss why we need team courses and present various ways to structure them. David Umphress, Dean Hendrix, and James Cross describe their long search for a suitable framework for teaching software project courses, and Lisa Burnell, John Priest, and John Durrett describe a unique and innovative approach to teaching in a simulated industrial environment.

W

e hope this special issue serves as a catalyst for continued discussion of society’s needs for safe, secure, and reliable software products and the importance of software education in addressing these needs.

References 1. Chaos, A Recipe for Success, The Standish Group Int’l, West Yarmouth, Mass., 1999. 2. Cutter Consortium, “78% of IT Organizations Have Litigated,” The Cutter Edge, 9 Apr. 2002, www.cutter.com/research/2002/edge020409.html. 3. “The Economic Impacts of Inadequate Infrastructure for Software Testing,” Planning Report 02-3, NIST, Gaithersburg, Md, 2002; http://nist.gov/director/ prog-ofc/report02-3.pdf.

About the Authors Watts Humphrey is a Fellow of the Soft-

ware Engineering Institute at Carnegie Mellon University. His principal areas of interest are in software process, process improvement, teamworking methods, and transitioning improved methods into general practice. He holds graduate degrees in physics and business administration and an honorary PhD in software engineering. He led the initial development and introduction of the Software Capability Maturity Model and developed the Personal Software Process as well as the initial versions of the Team Software Process. He is a member of the ACM, an IEEE Fellow, and a past member of the Malcolm Baldrige National Quality Award Board of Examiners. Contact him at the Software Eng. Inst., Carnegie Mellon Univ., Pittsburgh, PA 15213; [email protected].

Thomas B. Hilburn is a professor of com-

puter science at Embry-Riddle Aeronautical University and a visiting scientist at the Software Engineering Institute, Carnegie Mellon University. His research interests include software processes, formal specification techniques, and curriculum development. He is a member of the ACM and IEEE Computer Society, serves as the academic editor for the Forum for Advancing Software Engineering Education, is the software engineering editor for Computing Reviews, and is an editor for the ACM/IEEE-CS Computing Curriculum–Software Engineering project. Contact him at the Dept. of Computing, Embry-Riddle Aeronautical Univ., Daytona Beach, FL 32114; [email protected].

focus

educating software professionals

Developing Software Engineers at the C-130J Software Factory Richard Conn, Lockheed Martin and Kennesaw and Southern Polytechnic State Universities

ockheed Martin’s C-130J Avionics/Software Integrated Product Team (hereafter referred to as the IPT) creates software that runs a wide variety of systems on the C-130J aircraft. This team develops embedded safety-critical real-time air vehicle software and a ground-based data analysis system for aircraft analysis. The IPT operates within the infrastructure of the C-130J Software Factory,1 which consists of

L

The experience of new hires at Lockheed Martin’s C-130J Software Factory serves as a focal point in discussing how industry and academia must coordinate efforts to produce effective software engineers. 0740-7459/02/$17.00 © 2002 IEEE

Sun workstations and PCs networked to Web servers, a configuration management server, an aircraft simulator implemented in software, and laboratories are composed of the aircraft’s hardware mounted in equipment racks for easy access. A Web-based digital nervous system2 supports software engineering activities including data collection and metrics generation for software product and process evaluation. This IPT has educational needs as diverse as the roles of the C-130J aircraft. IPT activities cover many software development domains that address corporate, Federal Aviation Administration, and national and international military and civilian requirements. Many new hires, however, lack preparation for this environment. This article discusses the IPT’s diverse education and training needs, focusing on how to address shortfalls in conventional computer science and engineering education that result in mismatched expectations between the new hire and the company.

The C-130J Airlifter and its Software Factory To appreciate this article’s perspective, it’s essential to understand the product we create. Lockheed Martin rolled out the first production C-130 aircraft on 10 March 1955. Since then, Lockheed Martin has built more than 2,100 C-130s, and over 60 nations worldwide fly them in dozens of variations. C-130 aircraft ■ ■ ■ ■ ■ ■ ■ ■

Carry troops, vehicles, and armaments into battle Drop paratroopers and supplies from the sky Serve as airborne and ground refuelers Serve as flying hospitals and hurricane hunters Provide emergency evacuation and humanitarian relief Perform airborne early warning and maritime surveillance Operate in extreme conditions, from the Sahara deserts to the ice of Antarctica Have helped recover space capsules September/October 2002

IEEE SOFTWARE

25

Terminology As used here, new hire refers to people who recently joined the IPT fresh out of school. It does not refer to those who join the IPT with previous software engineering experience. Avionics/Software Integrated Product Team (IPT) refers to the team of people who develop the avionics and software for the aircraft and integrate the various aircraft systems into a cohesive whole. This team works in the Software Factory, an environment of networked computers, laboratories, and software and hardware simulators. Mature software engineering organization refers to maturity as defined by the Software Engineering Institute’s Software Capability Maturity Model. CMM Levels 4 and 5 designate mature software engineering organizations. Data primitives refer to raw data such as defect counts and cost measurements. Measurements refer to information items derived from the data primitives, such as the number of defects per lines of code. Problem space refers to a body of knowledge that an IPT or individual must possess to accomplish a mission. It can cover many knowledge domains.

In May 1992, Lockheed Martin delivered the 2,000th C-130, a C-130H. In September 1992, formal development of the C-130J began. Unlike its predecessors, the C-130J is a software-intensive system, employing modern avionics that have significantly improved its performance. By March 2001, the C-130J was flying with a full complement of embedded safetycritical, real-time air vehicle software and had set 50 world records in the National Aeronautics Association’s C-1-N (aircraft weighing between 132,276 and 176,368 pounds) and short take-off and landing (STOL) categories for speed, altitude, time-to-climb, and other aspects of the aircraft operation (see www.lmasc.com and www.lockheedmartin.com). The C-130J Airlifter incorporates distinct products, such as display panels, radars, and engines, from over 20 suppliers. Two mission computers backed up by two bus interface units integrate these products into a cohesive whole and control all aircraft functions. During operation, the C-130J Airlifter is supported by a ground-based data system that offloads data contained in the digital flight data recorders for analysis, including future failure prediction. The IPT develops the mission computer and bus interface unit software and the ground-based data system software, and integrates the supplier’s products. The IPT is supported by a Webbased digital nervous system that tracks problem reports, documents and tracks software changes, conducts software product evaluations, reports costs, and records and analyzes various process and product data. Problem spaces All new IPT hires have at least a bachelor’s degree in computer science or one of the engineering disciplines, such as electrical or mechanical engineering. They have written relatively small programs during their educational career, and, in virtually all cases, these programs had a well-defined problem space or domain. One of the first adjustments new hires face is realizing that they do not understand the entire problem space. Among some of the surprises:

26

IEEE SOFTWARE

September/October 2002

■

■

■

No one person completely understands all of the aircraft’s software. Teams in over 20 companies write the software, which the IPT then integrates into a cohesive whole. The various teams specialize in domains such as propulsion, radar, communications, air traffic control, and collision avoidance. Depending on the customer, even more specialized domains can exist, such as hurricane monitoring and atmospheric test equipment. Even within the domain of integrating the various aircraft systems, the software engineer does not know enough to just sit down and write the code. Software engineers work with systems engineers, customer representatives, flight test specialists, and others to draw up the requirements for the code they will later have to write. Software quality assurance specialists, auditors, and other team members check their work. To most new hires’ surprise, requirements specification is the single greatest source of injected defects in our code. The code is large, well beyond most new hires’ previous experience. The IPT constructs the C-130J airborne software baseline from which the IPT then derives customer-specific variants, and this adds up to over five million lines of code. The new hire’s experience with relatively small programs that cap off at 10,000 lines of code in a well-understood domain offers scant preparation for the more constrained discipline needed to build a larger, safety-critical software system like that on the aircraft.

College graduates as raw material This new environment comes as quite a cultural shock to most new hires, who, proud of their accomplishments at school, find that their education to date is really just a springboard for another step. Many companies view people straight out of school as raw material to be shaped to the corporate culture. In such a setting, new hires face some common frustrations. Process. A mature software development organization follows a defined process that, while mutable, does not change easily or quickly. Many ideas the new hire carries from

academia might not fit into the process and thus simply fall by the wayside. New hires may feel like they are taking a step backward, and in some cases they are—but for a good reason. The mature software development organization’s objective is to get a defined product out the door on schedule and within budget with no major defects and few minor defects. Unwisely introducing new technology and ideas can disrupt these deliveries. CMM Level 5 recognizes as key the transfer of new technology into the organization, but only when disruption will be minimal. The place of coding. Coding is often a very small part of a software engineer’s life, while in academia coding was probably the end-all and be-all of the new hire’s view. In a mature software development organization, much of a software engineer’s life is spent in meetings, discussing requirements, planning, evaluating software products (requirements specifications, designs, code, test scripts, deliverable documents, and more), documenting and reporting, and testing. The IPT’s software development process incorporates over 110 distinct subprocesses, only a handful of which involve writing code. Early team roles. Many new hires expect to jump into the “fun” coding work right away, but this is seldom the case. Before they move into the development groups that actually write the code, new hires are usually assigned to test code and participate in software product evaluations on code, requirements specifications, designs, test scripts, and other deliverables and nondeliverables. This is often more challenging and frustrating than coding. These early roles, however, serve an important purpose: catching defects before product delivery. These roles also serve an important secondary purpose in that new hires learn about the corporate culture and the people they will work with later as requirements engineers, designers, coders, and testers. Role-specific training for all IPT members is controlled by the Software Factory’s Web-based Learning Navigator tool and augmented by mentoring. Metrics. Providing various data primitives and

metrics to management, rather than just concentrating on the work at hand, forms a significant part of the corporate culture. Few new

hires have experience using metrics for project management or see the value in doing so. More mature software engineering organizations, by contrast, use data primitives gathered throughout their processes’ execution. They also use metrics computed from those data primitives to measure the quality of the products produced and processes executed by their teams. They establish “metrics yardsticks” by which to measure process and product quality and then use these to establish new goals. Metrics help senior management gain visibility into programs and projects for which they are responsible without having to take part in the day-to-day activities. Metrics help first- and second-line management gain insight into where defects are being injected into their processes and products, making it easier to improve the quality. Metrics are a way of life in mature software engineering organizations. Soft skills: A missing ingredient A common and significant omission in most new hires’ education is the development of “soft skills”—communication, sales and marketing, and teamwork in particular. One of the greatest shocks to a new hire is the amount of communicating a software engineer must do. Some of the communication software engineers perform routinely includes documenting designs, preparing and giving presentations, discussing and evaluating problems and challenges, understanding and evaluating software products, tracking action items, and documenting changes to software products. They also must write requirements, test plans and cases, defect reports on software products, and activity reports. Few new hires did much or any of this in school. Many new hires are surprised by the need for sales and marketing skills. Teams seldom accept new ideas readily from any team member, particularly a new hire. Team members must communicate their ideas effectively to sell both software engineers and management (which might have little or no background in software engineering) on them, and they must package and market them with increasing skill as they go further up the engineering and management chain. To new hires, the frequent rebuff of their ideas can be a constant source of frustration. Even experienced software engineers find this frustrating, but the better ones

Coding is often a very small part of a software engineer’s life, while in academia coding was probably the end-all and be-all of the new hire’s view.

September/October 2002

IEEE SOFTWARE

27

Unless our educational system can produce graduates familiar with and ready to adopt more mature software development practices, software failures will persist.

28

IEEE SOFTWARE

have the advantage of a track record that gives them an edge new hires do not enjoy. New hires’ education-oriented track records seldom count in these situations. Adjusting to working on a mature software development team poses several additional problems. Most new hires’ rewards throughout their educational career have come from their individual efforts, and they typically have little exposure to the team-oriented processes found in mature software development organizations. They might be used to meeting deadlines through individual heroic effort, often at the last minute, which contradicts a controlled, process-driven software development approach where deadlines are met through consistent team effort over time—sometimes months or years. Heroic efforts might prove necessary from time to time, but as both the software and systems engineering activities mature, such efforts become less frequent. New hires also might find software product evaluations discouraging. When a team critically examines the new hire’s software (such as a test plan or a requirements specification), the new hire might view such critiques as attacks on his or her work. Educational experience rarely prepares the new hire for a world in which software cannot be produced without teamwork and evaluation. This cultural mismatch creates vastly different expectations for new hires and the mature software development organization employing them. Retention and investments lost This mismatch of expectations can lead to problems in retaining new hires. The first job might become just a training ground for the next as the new hire becomes increasingly frustrated with the discipline of mature software engineering. When a company loses a new hire, it loses its investment in that person. Organizations must plan for such a loss, being ready to fill the gap until they can find a replacement. The safeguards in place in mature software engineering organizations ensure that the work the new hire has done is not lost, but nothing can replace the initial investment. IPT Solutions Overcoming one culture and instilling a different one in a group of people can be difficult and time-consuming. As happens at most companies, new IPT hires attend a

September/October 2002

new employee orientation, a sort of preliminary indoctrination to the culture. This indoctrination covers a lot of material, and time constraints prevent in-depth explanations of why things are the way they are. New hires then go to their specific assignments, where they receive job-specific material to study and are assigned mentors to guide them in their daily work. They also receive role-based training, which offers the opportunity to explore the rationale behind company processes and procedures. However, most new hires don’t ask such questions, simply accepting the training at face value. We therefore try to work into the training information on the rationale. What universities can do Being educational rather than training institutions, universities and colleges produce graduates whose educational foundation lets them technically adapt to challenges both foreseen and unforeseen. Corporations like Lockheed Martin then take the new hires as raw material, training them to deal with organizational issues such as ■

■

■

Software process and its execution—how to produce requirements specifications, create designs, develop code, evaluate software products, test software products against the requirements, and so on Appropriate use—how to use the organization’s software tools and programming languages within the bounds of the processes and methodologies adopted Organization member information— such as chain of command, problem reporting, and daily operation details

The IPT interacts with local universities, particularly Southern Polytechnic State University and Kennesaw State University, and such institutions as the Software Productivity Consortium (SPC; www.software.org) and the Software Engineering Institute (SEI; www.sei.cmu.edu), to help address its employees’ educational needs. The resulting cultural change we see in new hires from these universities—from a focus on coding to involvement in other software engineering activities such as requirements definition—eases their entry into the corporate culture. We would see more of this across the industry if universities would interact

About the Author Richard L. Conn is

more with local firms, collaborate with organizations like the SPC and SEI, and become involved with groups such as the Forum for Advancing Software Engineering Education (www.cs.ttu.edu/fase) to help faculty better understand the issues facing industry. At the IPT, organization members provide some training; other training comes from Southern Polytechnic State University’s Software Engineering Retraining Program (http://cs.spsu.edu) and the Software Productivity Consortium’s courseware collection, among others. Such training, however, is not appropriate as part of a regular degree program at a local university or college—an institution would compromise graduates’ educational foundation if it permitted such corporate influence in its degree programs. More fitting are courses like CSIS 1020 at Kennesaw State University (http://science.kennesaw.edu/csis/), which introduces Visual Basic programming with a software engineering flavor to first-year students, and degree programs like Southern Polytechnic State University’s master’s in software engineering (http://cs.spsu.edu), which covers general concepts such as capability maturity and software engineering principles. Rather than making training specific to the current needs of local industry part of their degree programs, universities and colleges should foster fundamental cultural change by adding software engineering and soft-skill knowledge units. Such knowledge units might include ■

■ ■ ■

■ ■

Dealing with problems larger than a single student can grasp and that require a team of domain specialists to solve Working with a team of people under the control of a process Developing products that a team of peers will critically evaluate Testing and evaluating products effectively, to detect as many defects in the products as possible, and learning how to deal with the critical evaluation of one’s own products Communicating concepts effectively and selling and marketing ideas to others Measuring product and process quality analytically, establishing criteria to assess product or process improvements

Such cultural changes can better position our graduates to produce the higher-quality products we need now and in the future. Work was already underway to place software engineering knowledge units into Computing Curriculum 2001 of the Association for Computing Machinery’s Special Interest Group in Computer Science Education (www.acm.org/sigcse/cc2001/), and software engineering degree accreditation under the auspices of the Accreditation Board for Engineering and Technology (www.abet.org) will begin in Fall 2002, but it must go further than just covering software engineering topics to effect the desired cultural change.

A

s society relies increasingly on software for critical operations, software failures become intolerable. In aircraft, they can cost lives; in our communications systems, they can cost billions of dollars. In a microwave oven, a software failure can result in a very costly recall of thousands of units. Failure of a commercial software product can take its manufacturer out of business, and if a Web site used to transact sales fails, a company could lose millions of dollars an hour until the Web site is repaired. Unless our educational system can produce graduates familiar with and ready to adopt more mature software development practices, however, software failures will persist. The education community must evolve at the cultural level to produce graduates who already embrace mature software engineering practices. While students acquire knowledge relatively easily, the wisdom to apply that knowledge well comes much more slowly. The cultural changes proposed here, therefore, go beyond development of technical knowledge to culturing the experience and mindset needed to create and apply high-quality software.

a software process engineer for the C130J Airlifter at Lockheed Martin Aeronautics Company. His research interests include software process engineering, component-based software engineering, software reuse, and software engineering education. He holds a BS from Rose-Hulman Institute of Technology and an MS from the University of Illinois, both in computer science. He has served on the Federal Advisory Board for Ada (receiving ACM/SIGAda’s award for Outstanding Contributions to the Ada Community), the DoD Software Reuse Initiative, ACM and IEEE software engineering education workshops, and IEEE standards efforts. Contact him at Lockheed Martin Aeronautical Systems, 86 South Cobb Dr., Dept. 70-D6, Mail Zone 0674, Marietta, GA 30063-0674; [email protected], http://unicoi. kennesaw.edu/~rconn, http://cs.spsu. edu/rconn.

References 1. R. Conn, S. Traub, and S. Chung, “Avionics Modernization and the C-130J Software Factory,” Crosstalk— J. Defense Software Engineering, vol. 14, no. 9, Sept. 2001, pp. 19–23. 2. B. Gates, Business @ the Speed of Thought, Warner Books, New York, 1999; www.speed-of-thought.com.

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

September/October 2002

IEEE SOFTWARE

29

focus

educating software professionals

A Collaborative Program to Retrain Lockheed Martin Aero Engineers Jorge Díaz-Herrera, Mike Murphy, and Dawn Ramsey, Southern Polytechnic State University

ockheed Martin Aeronautics Company in Marietta, Georgia, wanted to transform a surplus of engineers from traditional areas into software engineers, whom it found hard to attract and keep as employees. LM Aero-Marietta approached Southern Polytechnic State University to plan and offer a software engineering retraining program.

L

Planning In early 1999, LM Aero-Marietta asked SPSU to investigate these concerns:

This innovative program turns engineers from traditional areas into software engineers through university coursework and a company practicum. Others considering such partnerships will benefit from reading about the program’s planning and implementation, results thus far, and the organizers’ recommendations. 30

IEEE SOFTWARE

■

■

■

High turnover in software engineering positions due to competition for such professionals Significant downsizing that included personnel with valuable domain skills in traditional engineering and avionics Continued viability depending, in part, on filling critical software engineering positions

The resulting investigation was two-fold: to establish the specific needs associated with these concerns and to determine the feasibility of tailoring a program to address these needs. The preliminary investigation took place in the spring and summer of 1999. Initial conversations between LM Aero-Marietta managers and members of SPSU’s Computer Science Department (now the School

September/October 2002

of Computing and Software Engineering) dealt with the company’s overall software engineering environment and SPSU’s capabilities to respond effectively to the results of a full investigation. The Computer Science Department had long offered strong undergraduate and graduate programs in computer science and had a major interest in software engineering since the late 1980s. It added a professional master of science in software engineering (MSSWE) degree in 1997. In August 1999, we agreed to proceed with a formal needs assessment. The assessment included focused interviews with key software managers at LM Aero-Marietta, followed by written responses to a survey that addressed numerous software engineering issues. The survey was based on IEEE standards related to software engineering (http://standards.ieee. org/software). The resulting report proposed an undergraduate credit-based certificate, the Software Systems Development Certifi0740-7459/02/$17.00 © 2002 IEEE

cate, to be delivered at SPSU to LM AeroMarietta personnel. In addition to the 32credit coursework to be delivered on an accelerated schedule, the SSDC required a 12-week, 240-hour internship at the company. Another possible component, providing professional growth and development for existing LM Aero-Marietta software engineers, was the Graduate Certificate in Software Engineering, an existing creditbased certificate program that provides one half of the coursework required for the MSSWE. We decided to use the new undergraduate certificate as the primary response. Funding The state of Georgia offers partial funding for economic development initiatives. One program, the Intellectual Capital Partnership Program (www.icapp.org), is designed to build partnerships among the state, business and industry, and public higher education. ICAPP’s purpose is to help companies meet specific workforce needs in Georgia for educated workers in high-demand, high-technology fields. Georgia provides focused funding, SPSU provides expertise, and LM Aero-Marietta agrees to hire the program’s graduates for a minimum of two years. In addition, participants are eligible for Georgia’s Hope Grant (as opposed to the Hope Scholarship for high-achieving traditional students) to cover tuition, fees (except parking), and books. Our ICAPP proposal was submitted in June 2000, funded in August 2000, and the first group of students started in November 2000. LM Aero-Marietta pays the students their regular salaries during the 32 weeks, which represents a major commitment by the company. Georgia (through ICAPP) pays for a computer classroom, faculty stipends, and administrative support costs, and SPSU provides the program space and instruction. We wanted to immerse the students in a campus environment (SPSU is approximately a mile from the company); the courses are credit-bearing, and distractions would be fewer. The initial version of the SSDC required 10 credit-based courses that would lead to a meaningful student transcript representing career-focused education rather than just skills acquisition. Other factors that we addressed included a workable format that would produce results over a relatively tight timetable,

selection of participants, and program funding. The proposed format was full days, 8 am to noon and 1 pm to 5 pm, for 30 weeks (the extra two weeks mentioned earlier include minor breaks and official holidays). In general, the morning is devoted to instruction and the afternoon to lab work and study. After 18 weeks, the afternoon lab ends and students begin a 12-week internship. This makes their days quite full, as the assignments and projects for the courses don’t diminish during the internship. Curriculum The SSDC curriculum has evolved over time, and has stabilized to include a twocourse sequence on programming fundamentals, including the basics of the Personal Software Process,1 and introductory twocredit courses on software engineering, databases, and operating systems. These are followed by more advanced courses, including client-server programming, Ada software development, real-time systems, software testing and quality assurance, and a software engineering project. The project course covers advanced Ada topics in a team project setting and is supported by the Team Software Process.2 It requires students to make extensive use of graphics libraries as well as tasking and distributed clientserver processing, all within Ada. The last course is software project management, which serves to tie up loose ends and gives a broad perspective of the development process. One of the challenges in building and adjusting the curriculum was to be sure that students have sufficient background for a meaningful internship while still taking courses in the program. As soon as the proposal was approved, faculty were sought, and they began adapting regular courses to the compressed format. Faculty members received stipends for developing new courses, revising existing courses into this format, and teaching them. The stipend was used as an incentive and to minimize the disruption to our regular offerings. We hired an administrative support person, identified space for the computer classroom, ordered equipment, and began searching for a half-time lab support technician. While this was happening, we started soliciting and reviewing applications of potential participants.

Georgia provides focused funding, SPSU provides expertise, and Lockheed Martin Aeronautics agrees to hire the program’s graduates for a minimum of two years.

September/October 2002

IEEE SOFTWARE

31

Performance is best overall for those who come in with some software development experience but not too much.

Implementation Coordination of the SPSU academic program centered on a faculty advisory committee composed of four professors and the CS Department industry liaison. Committee members visited the company to present the program’s curriculum to potential participants and answer questions. Others from SPSU and LM Aero-Marietta were also present at these information sessions to explain admission to SPSU as well as the selection and processing of applicants. The committee developed an evaluation form to assess the academic indicators of program success (for example, math maturity, previous GPA and degree type, and motivation as indicated in a statement of purpose in applying). Applicants were screened into one of three categories: highly recommended, acceptable, and not recommended. The files of those in the top two categories then went to LM Aero-Marietta, where managers of software-intensive units reviewed them. The managers interviewed the candidates who met the company’s desired profile (for example, work history, domain knowledge, and references) and chose the top applicants. The plan was to have four staggered groups of 20 each. Results to date The first three groups have graduated. The students refer to themselves as “SERPents,” showing they can maintain a sense of humor and camaraderie while being immersed in an intense retraining program. There are some interesting observations that we can make at this point. The students tend to have some difficulty adjusting to the rapid pace and academic nature of the program. They are used to having rather well defined requirements and deliverables for their work as engineers. Thus, they are rather intense and anxious about grades and course requirements. They sometimes have surprisingly high expectations for the instruction they receive, and there is the adjustment from one instructor to the next. The first group objected to the term “internship” as somewhat demeaning for those who have already been practicing engineers, so we changed it to “practicum,” which has been well received thus far. Also, some managers had unrealistic expectations regarding the students’ software

32

IEEE SOFTWARE

September/October 2002

engineering maturity at the point of the practicum. We have done some tuning of the courses based on feedback from students and managers. In particular, we increased the coverage of software development in Ada. We also lengthened the delivery time of the two-credit courses by running them in parallel at half the contact time each for twice the number of days, because the original format caused that material to be frustrating in terms of time to absorb and apply concepts. We also found that performance is best overall for those who come in with some software development experience but not too much. Too little or none tends to cause anxiety at the pace of instruction, and too much brings up issues of unlearning bad practices and resisting new or more formal approaches because “that isn’t the way it gets done in real life.” From the other side, managers also have concerns about taking on a senior engineer (from a traditional field) as a trainee in software engineering. Figure 1 shows how the third delivery of the program was organized; the top part of the figure describes lectures and the bottom part lab and practicum time. Originally, the software engineering project course was placed toward the end of the program, because students would be doing their practicum at the same time (in the afternoons) and could address problems with real systems. This, however, caused severe problems, as we explain later, so we decided to place the project course before the practicum. This worked much better, because the project course ■

■

■

Immediately followed the Ada programming course, allowing smooth continuation of this material Exposed students to structured teamwork, which turned out to be very useful for application during the practicum Gave students a higher-level view of real-time programming and distributed computing before they were exposed to the lower-level details of these topics

We planned the project course around the Introductory TSP framework, which is designed for a semester of 15 to 17 weeks. However, the course had to be taught in

PSP (light) Programming Fundamentals I and II C++

150 hours

TSPi

Intro to Software Engineering Intro to Operating Systems

Application Programming Ada

60 hours

Software Engineering Project

Ada (advanced)

60 hours

75 hours

Laboratory (afternoon)

Real-Time Systems ClientServer Quality Intro to Assurance Databases

150 hours

Project Management

90 hours Practicum (afternoon)

Figure 1. Program organization for the third and fourth groups.

four weeks, so drastic changes were in order. TSPi calls for a cyclical development approach, each of the three cycles preceded by a launch (or relaunch) and followed by a postmortem. The first time we taught the course, we condensed two cycles, and this did not work well. The amount of record keeping necessary to collect all TSPi data and the learning curve of the process scripts, forms, and such was overwhelming for the amount of time available—compounded by the fact that the students did not have the afternoon lab time anymore because they were engaged in the practicum. The second and subsequent offerings of the course focused on one cycle and a modified implementation phase organized in builds or minicycles. We had only one project launch and one postmortem. The instructor and the students were very satisfied with the results. The curriculum was not the only thing that changed. The company’s commitment diminished due to evolving needs, changes in the general economy, and winning or losing specific government contracts. The third group included only 10 LM Aero-Marietta employees plus two people from elsewhere. The program will end when the fourth group graduates in December.

A

lthough others have developed software engineering retraining programs,3–5 the program presented here reflects a collaborative effort of higher

education, government, and industry responding to a critical need in a specific domain, aerospace. The lessons learned are similar to those from software development in general: the field is young and immature; requirements change almost as fast as they can be written down; process improvement is at the heart of effective and efficient delivery; and what works on one project doesn’t assure that it can be replicated. Anyone interested in keeping up with software engineering education, training, and retraining should consider attending the annual Conference on Software Engineering Education and Training (CSEE&T). Based on our experience to date, the following key principles have emerged: ■ ■

■ ■

Leverage support for economic development from seemingly diverse sources. Nurture the relationship with those committed to the program’s vision and success. Establish and maintain high levels of communication with all parties. Be flexible in building such programs but do not abandon core values.

These “platitudes” are critical to success in linking higher education to the needs and support of business and industry. The retraining program at SPSU is making a difference. Lockheed Martin has publicized the program widely and is proud of participating and adding value to their workforce.6 September/October 2002

IEEE SOFTWARE

33

About the Authors

References 1. W. Humphrey, Introduction to the Personal Software Process, Addison-Wesley, Reading, Mass., 1997. 2. W. Humphrey, Introduction to the Team Software Process, Addison-Wesley, Reading, Mass., 2000. 3. A. Ben-David et al., “An Industrial Software Engineering Retraining Course: Development Considerations and Lessons Learned,” IEEE Trans. Software Eng., vol. 10, no. 6, June 1984, pp. 748–755. 4. J.Z. Loeb (Lavi), M.I. Ben-Porat, and A. Ben-David, “IAI Corporate Software Engineering Training and Education Program,” IEEE Trans. Software Eng., vol. 13, no. 11, Nov. 1987, pp. 1207–1216. 5. “Carleton Computer Science Certificate (Software),” Carleton Univ. School of Computer Science, www.scs.carleton.ca/~dehne/proj/certificate. 6. “Programmed for Success,” Lockheed Martin Today, vol. 7, no. 8, Aug. 2001, p. 4.

Jorge Díaz-Herrera is dean and a professor in the Golisano College of Computing and

Information Sciences at the Rochester Institute of Technology. Formerly, he was a professor of computing and software engineering at Southern Polytechnic State University, where he participated in the work for this article. He has also worked at Carnegie Mellon University’s Software Engineering Institute. Some of his current activities relate to software engineering education and the software engineering of embedded systems. Contact him at the Golisano College of Computing and Information Sciences, Rochester Inst. of Tech., 20 Lomb Memorial Dr., Rochester, NY 14623-5603; [email protected]. Mike Murphy is founding dean and professor in the School of Computing and Software

Engineering at Southern Polytechnic State University. He is interested in software engineering education, courses, and curricula. Contact him at Southern Polytechnic State Univ., School of Computing and Software Eng., 1100 S. Marietta Pkwy., Marietta, GA 30060-2896; mmuphy@ spsu.edu.

Dawn Ramsey is dean of the Extended University at Southern Polytechnic State University and industry liaison for the School of Computing and Software Engineering at SPSU. Her areas of interest are industry–academia collaboration and retraining programs. Contact her at the Extended Univ., Southern Polytechnic State Univ., 1100 S. Marietta Pkwy., Marietta, GA 30060-2896; [email protected].

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

COMPUTER

PURPOSE The IEEE Computer Society is the world’s

MEMBERSHIP Members receive the monthly magazine COM PUTER , discounts, and opportunities to serve (all activities are led by volunteer members). Membership is open to all IEEE members, affiliate society members, and others interested in the computer field.

BOARD OF GOVERNORS Term Expiring 2002: Mark Grant, Gene F. Hoffnagle, Karl Reed, Kathleen M. Swigger, Ronald Waxman, Michael R. Williams, Akihiko Yamada Term Expiring 2003: Fiorenza C. AlbertHoward, Manfred Broy, Alan Clements, Richard A. Kemmerer, Susan A. Mengel, James W. Moore, Christina M. Schober Term Expiring 2004: Jean M. Bacon, Ricardo Baeza-Yates, Deborah M. Cooper, George V. Cybenko, Wolfgang K. Giloi, Haruhisha Ichikawa, Thomas W. Williams Next Board Meeting: 8 Nov 02, Boston, MA.

IEEE

OFFICERS

President: RAYMOND D. FINDLAY

EXECUTIVE COMMITTEE President: WILLIS K. KING* University of Houston Dept. of Comp. Science 501 PGH Houston, TX 77204-3010 Phone: +1 713 743 3349 Fax: +1 713 743 3335 [email protected] President-Elect: STEPHEN L. DIAMOND* Past President: BENJAMIN W. WAH* VP, Educational Activities: CARL K. CHANG * VP, Conferences and Tutorials: GERALD L. ENGEL* VP, Chapters Activities: JAMES H. CROSS VP, Publications: RANGACHAR KASTURI VP, Standards Activities: LOWELL G. JOHNSON †

†

(2ND VP)*

VP, Technical Activities:

DEBORAH K. SCHER-

President-Elect: MICHAEL S. ADLER

RER(1ST VP)*

Past President: JOEL B. SYNDER

Secretary: DEBORAH M. COOPER* Treasurer: WOLFGANG K. GILOI* 2001–2002 IEEE Division VIII Director:

Executive Director: DANIEL J. SENESE Secretary: HUGO M. FERNANDEZ VERSTAGEN Treasurer: DALE C. CASTON VP, Educational Activities: LYLE D. FEISEL VP, Publications Activities: JAMES M. TIEN VP, Regional Activities: W. CLEON ANDERSON VP, Standards Association: BEN C. JOHNSON VP, Technical Activities: MICHAEL R. LIGHTNER President, IEEE-USA: LeEARL A. BRYANT

SOCIETY

WEB

SITE

The IEEE Computer Society’s Web site, at http://computer.org, offers information and samples from the society’s publications and conferences, as well as a broad range of information about technical committees, standards, student activities, and more.

largest association of computing professionals, and is the leading provider of technical information in the field.

THOMAS W. WILLIAMS

2002–2003 IEEE Division V Director: GUYLAINE M. POLLOCK†

Executive Director:

DAVID W. HENNAGE†

*voting member of the Board of Governors

COMPUTER SOCIETY O F F I C E S Headquarters Office 1730 Massachusetts Ave. NW Washington, DC 20036-1992 Phone: +1 202 371 0101 • Fax: +1 202 728 9614 E-mail: [email protected] Publications Office 10662 Los Vaqueros Cir., PO Box 3014 Los Alamitos, CA 90720-1314 Phone: +1 714 821 8380 E-mail: [email protected] Membership and Publication Orders: Phone: +1 800 272 6657 Fax: +1 714 821 4641 E-mail: [email protected] European Office 13, Ave. de L’Aquilon B-1200 Brussels, Belgium Phone: +32 2 770 21 98 • Fax: +32 2 770 85 05 E-mail: [email protected] Asia/Pacific Office Watanabe Building 1-4-2 Minami-Aoyama, Minato-ku, Tokyo 107-0062, Japan Phone: +81 3 3408 3118 • Fax: +81 3 3408 3553 E-mail: [email protected]

EXECUTIVE

STAFF

Executive Director: DAVID W. HENNAGE Publisher: ANGELA BURGESS Assistant Publisher: DICK PRICE Associate Executive Director: ANNE MARIE KELLY Chief Financial Officer: VIOLET S. DOAN Director, Information Technology & Services: ROBERT CARE Manager, Research & Planning: JOHN C. KEATON 12-JULY-2002

34

IEEE SOFTWARE

September/October 2002

focus

educating software professionals

Software Engineering Programs: Dispelling the Myths and Misconceptions Hossein Saiedian, University of Kansas Donald J. Bagert, Rose-Hulman Institute of Technology Nancy R. Mead, Software Engineering Institute

n a now classic 1994 Scientific American article, W. Wayt Gibbs described software crises in both the private and government sectors.1 The problems he discussed ranged from overrunning budgets and schedules to terminating projects despite multimillion-dollar investments. Similar concerns were reported as recently as March 2001 in the Communications of the ACM, where several authors made grim predictions about the future of software engineering (SE) if the industry continues with “business as usual.”2

I Some people think new software engineering degree programs address industrial software development problems; others argue that they are merely an opportunity to provide industrial training in programming. The authors address these and similar issues, discussing commonly held myths about such programs. 0740-7459/02/$17.00 © 2002 IEEE

In recent years, software researchers and developers have explored numerous principles for improving software practices, some of which have proven effective in practical projects. These include software development methodologies and environments, structured and object-oriented programming, software process improvement models (such as the CMM), Computer-Aided Software Engineering (CASE) tools, and fourth-generation languages. Nevertheless, we have not completely resolved our software problems, and many organizations continue to suffer from bad practices. One way to improve practice is to focus on properly educating the next generation of SE professionals. However, the debate about the most effective approach to educating this next generation continues unabated. Some argue for an SE track under existing computer science (or computer en-

gineering) programs. Others promote specialized and independent SE degrees at the graduate as well as undergraduate levels. Some universities have established such specialized programs, hoping that they will address all industrial SE problems. Unfortunately, universities often don’t know what new programs should offer or when or at what level they are most appropriate. Furthermore, traditional computer scientists have criticized these programs as merely providing an opportunity to offer industrial training in programming. In many ways, the current situation mirrors that of the computer science field in the 1960s and 70s: electrical engineering and mathematics faculty initially resisted the growth of computer science degree programs just as current computer faculty are treating SE today. Contributing to the hype about and harsh criticism of SE programs are some widely held September/October 2002

IEEE SOFTWARE

35

Adding a new degree program will lead to a better image, but we must calculate the associated costs to determine whether the program will be cost effective in the long run.

myths. This article examines these myths, dispelling the misconceptions to defuse unnecessary concerns, conflicts, and distractions and help provide an appropriate context and direction for new SE degree programs. Myth 1 A new software engineering degree program is an academic necessity.

There seems to be a rush to develop new SE degree programs (especially at the graduate level) simply because many SE faculty members believe that adding such programs will improve the reputation of their departments and institutions. Adding a new degree program will lead to a better image, but we must calculate the associated costs to determine whether the program will be cost effective in the long run. In certain market areas, such programs are absolutely necessary and a welcome addition; in other areas, they won’t have such an impact and thus might become a burden. Institutions should not develop new programs based on image-enhancing effects, popular trends, or peer pressure. Rather, they should introduce a program only if it is necessary and has a valid and viable market. Those interested in introducing a degree program must objectively assess existing industrial needs, the potential pool of students, expertise among the existing faculty, and administrative support. Furthermore, developing a new SE program is not always the only means of addressing the needs of local industrial organizations or student requests. Many times, adding core SE courses in a computer science program or adding an SE focus area (or specialization) will address both real and perceived demands. Myth 2 Software engineering programs will unnecessarily expend computer science resources.

The general consensus in both industry and academia is that computer science degree programs are worthwhile and should continue for the foreseeable future. Therefore, to meet the demand for computer scientists as well as the emerging need for those educated in SE, there must be sufficient resources (especially faculty) for both programs. There is a severe shortage of faculty in all computing fields, including SE. Master’s 36

IEEE SOFTWARE

September/October 2002

programs in SE often overcome this obstacle by using part-time adjunct faculty, but this is a less viable option for many undergraduate programs. With a small pool of potential new faculty available, some institutions must retrain computer science faculty to teach SE. However, finding people willing to undergo such retraining is more difficult than in industry, due to the rights that tenured faculty exercise at many academic institutions. The discussion here assumes that a computer science department will house the SE program, perhaps becoming the Department of Computer Science and Software Engineering (CSSE). Because the disciplines are closely related, housing them in the same academic unit is the best option, letting the two faculties work together for their mutual good. However, at many institutions, there are difficult political issues involved in forming a CSSE department, especially if the Computer Science Department is not in a College of Engineering. Furthermore, people believe that an SE program in such a CSSE department would considerably drain computer science resources— but this is not necessarily the case. Consider the following scenario: The curriculum for a computer science department in a College of Engineering has nearly half of its credit hours in the computer science department. In addition, there is a 90/10 percent split between the computer science and SE hours, which is consistent with the core material recommended for a computer science degree under Computing Curricula 2001 (CC2001).3 Thus, 90 percent of the computer science faculty must teach computer science courses, and the other 10 percent must teach software engineering classes. Now suppose the department adds an SE degree program. The total number of credit hours computer science and SE majors take through the newly named CSSE Department are the same, but the split in hours is now 50/50 between the two disciplines. (This is consistent with the recommendations made in the Guidelines for Software Engineering Education,4 which provides the SE undergraduate curriculum model most cited in recent literature.) Subsequently, one-third of the department’s majors are in SE and the other two-thirds are in computer science. How does this affect the allocation of resources?

When academic programs discuss resources, they’re generally referring to faculty. Suppose the Computer Science Department has 30 faculty members. Before the SE degree existed, the department would need 27 computer science instructors and three SE instructors, due to the 90/10 percent split in hours. Under this scenario, implementing the SE program would require a shift of four faculty members. The computer science majors (twothirds of the total) would require 18 computer science and two SE instructors; the SE majors would need five computer science and five SE instructors, meaning that the computer science and SE split of the faculty would now be 23/7. So, only 13 percent of the faculty (four out of 30) would need to shift to SE to implement the change. Considering the number of electrical engineering and mathematics faculty that changed to computer science as it was emerging, this seems reasonable, despite the retraining issues involved. Furthermore, it is likely that students who would have originally been computer engineering majors in an Electrical and Computer Engineering (ECE) Department will now be SE majors. Those students will take about the same number of computer science hours as in computer engineering, so there will be no additional drain on computer science faculty; any additional faculty needed would be in SE. So, this means that the department would gain more majors (at the expense of ECE) and (if available) more faculty to teach them. We can vary the scenario, but it still indicates that creating an SE program in an CSSE Department will have minimal negative impact on the faculty. The positive benefits— more majors, faculty, and choices available to the student—far outweigh the disadvantages, and we would get better-educated software professionals without depleting the supply of computer science graduates. Myth 3 Software engineering undergraduate programs do not have enough depth.

Many companies believe that a master’s degree program in SE provides a sufficient overall background for future software professionals. Such programs typically require the student to have a minimal background

of a set of undergraduate computer science courses such as data structures, discrete structures, design and analysis of algorithms, and operating systems, for a total of 24 to 30 hours. The question is, can an undergraduate SE program provide the computer science background needed while providing the additional SE topics necessary to educate a software professional? Several sources indicate the type of core computer science background that should be required. Tim Lethbridge surveyed software professionals and reported that the 25 most important topics required of such individuals include computer science areas such as specific programming languages, data structures, object-oriented concepts, design of algorithms, operating systems, systems programming, databases, file management, and networks.5 Providing undergraduate education in these topics would once again require 24 to 30 semester hours. This range of hours would also be sufficient for CC2001’s core requirements and many of the requirements for computer science courses (outside of SE) specified in the criteria of the Computing Accreditation Commission of the Accreditation Board for Engineering and Technology.6 (ABET is the accreditation body for engineering degree programs in the US.) Therefore, it is reasonable to assume that those same 24 to 30 computer science hours would be sufficient in an undergraduate SE curriculum. As far as determining how many SE credit hours are required, a typical master’s degree requires at least 24 semester hours of graduate course work in the major (SE, in this case). The question is, how does this translate to undergraduate hours? One option is to use a 3:2 ratio between undergraduate and graduate hours, which is commonly done in comparable courses in computer science and other disciplines. So 24 graduate hours would then translate to 36 undergraduate hours. It is possible (although difficult) to squeeze 24 to 30 hours of computer science and 36 hours of SE into an undergraduate SE curriculum. However, such comparisons are usually made using graduate courses that build on undergraduate classes in the same discipline, whereas a master’s degree in SE requires little or no SE background when entering the program. Therefore, 36 undergraduate SE hours might be too many.

Many times, adding core SE courses in a computer science program or adding an SE focus area will address both real and perceived demands.

September/October 2002

IEEE SOFTWARE

37

There must be a paradigm shift in attitude at the workplace. Existing software development professionals and managers must value and respect SE education.

The Guidelines for Software Engineering Education suggests that an undergraduate SE curriculum have 21 required hours of computer science, 24 required hours of SE, and nine hours of electives in either computer science or SE. That model was intended to satisfy CC2001 core computer science requirements and ABET criteria for SE undergraduate degree programs in the US. It also aims to cover the same material typically found in an SE degree program, all in a 120-semester-hour, four-year curriculum, which is generally the minimum requirement for a baccalaureate program in the US. (It is also interesting to note that virtually all of the 25 most important topics for a software professional cited in the Lethbridge survey— including those in computer science and SE—are also covered in the model from the Guidelines report, even though the latter was published first.) Such an SE curriculum would provide minimally sufficient depth for an SE major; allowing more than 120 hours (which is often the case for US engineering programs) would provide even more depth in computer science or SE. Myth 4 A new SE degree will address industrial software development crises.

A new SE degree program will not be a panacea or a silver bullet. It will be one of the first steps, albeit the most important step, toward addressing industrial software development crises, but we also must consider complementary factors. For example, we need to clearly define the “engineering education” (the curricula and the style of presentation). A starting point is understanding the objective of an engineering education as David Parnas defines it.7 He argues that to provide the most effective SE education, a new SE program must follow the traditional engineering approach to professional education while maintaining the scientific basis of SE (computer science). He emphasizes that an engineering education should teach engineers ■ ■ ■

38

IEEE SOFTWARE

What is true and useful in their chosen specialty How to apply the body of knowledge How to apply a broader area of knowledge to build complete products that must function in a real environment

September/October 2002

(In a related article, Mary Shaw compares the evolution of chemical engineering and civil engineering with today’s SE. She uses her evolution model of engineering disciplines to identify the steps for enhancing the SE discipline.8) Another important issue is that of defining an acceptable body of knowledge for software engineers. The Software Engineering Body of Knowledge (SWEBOK) has been an excellent starting point, but it has certain deficiencies. One is its perceived North American bias; such a guide must obtain international acceptance. Yet another criticism surrounds its certification and licensing implications. We must clearly address such issues and invite other computing associations to join SWEBOK’s development. Furthermore, software engineers must be able to apply the methods in different contexts and tune their knowledge to more effectively use the new technologies. As Michael McCracken has observed,9 academia cannot predict the next popular language or methodology industry will use, so the education provided to software engineers must focus on the fundamentals to prepare the new graduates to assimilate and apply new technology quickly and efficiently. A long-term aspiration would be to identify distinct roles in SE and provide appropriate education and specialized training for each.10 SE students (and education) must include an element of training, not only during their academic careers (for example, through internships) but once they enter the workforce and before important design and implementation responsibilities are delegated to them. This is not only true in other engineering disciplines but also in nonengineering disciplines. (For example, in the medical field, new graduates go through at least two years of training as part of their residency program before they are allowed to engage in real practice.) Steve McConnell and Leonard Tripp suggest at least four years of apprenticeship for software engineers.11 In addition, there must be a paradigm shift in attitude at the workplace. Existing software development professionals and managers must value and respect SE (and computer science) education, acquaint themselves with the fundamentals and the body of knowledge, and update their personnel skills to avoid “cultural clashes” with newcomers. Otherwise, new SE graduates will be

unsuccessful in transitioning their new body of knowledge and will end up following unproven skills and the “code and fix” culture that dominates the workplace.11 Certification and licensing are equally important. Although formal education is crucial, a software engineer should also regularly (for example, every five years) prepare for and pass certification exams. This would assure that he or she has maintained a minimum understanding of the SE body of knowledge. Certification exams can then evolve into a kind of licensing exam, similar to licensing in other fields, to facilitate and assure professional competency and responsibility. Myth 5 Computer science is to software engineering what chemistry is to chemical engineering.

Another widely held misconception pertains to the relationship between computer science and SE (which has been compared with that of chemistry to chemical engineering or physics to mechanical engineering). This is essential in understanding how educating software professionals differs from educating computer scientists. This myth is tempting for SE faculty because it supports their contention that, over time, computer science will become more theoretical. In the long term, it will thus be difficult to find (more) room for SE in computer science curricula. The alternative then is to also develop SE curricula, which can focus more on the practical aspects of software development while also including topics such as software management, process, and project organization throughout the curriculum. It is true that computer science—itself a relatively young field—has gradually expanded both its theoretical and scientific bases, and that this has caused an increase in theory content in many computer science curricula. However, physics and chemistry are examples of physical sciences, whereas software is a nonphysical entity. As such, software (on a small scale, which is useful in an educational setting) can be easily created and duplicated. So, the development and manipulation of software should continue to be a central theme in computer science curricula as well as in practice by computer professionals.

Myth 6 Software engineering graduates will not need further training to perform like experienced software engineers.

Typical new SE graduates find themselves working on teams where they are expected to perform (with little or no additional training) alongside experienced software engineers. Organizations often assume that new hires can internalize corporate culture and standards on their own and acquire domain expertise on the job. The new hires often find themselves on a software project’s critical path. One reason for this is that most corporate managers think their new hires know the latest and greatest methods and can handle more challenging assignments than some of the folks who have been around for a while. Another reason is that the new hire is often expected to put in many extra hours and not to have outside family obligations. A third reason is that many software projects start out with difficult schedules, and it is just not feasible to give new staff members the time to gradually ramp up the learning curve. In Death March, Ed Yourdon says, “To many grizzled veterans... every project is a death march project.”12 Tom DeMarco, in his book Slack, says: “…a dangerous corporate delusion: the idea that organizations are effective only to the extent that all their workers are totally and eternally busy.”13 Consider the following example: A new hire joins a project in progress, replacing another employee who has been transferred to a different project. After a brief orientation, he or she inherits the other employee’s work and is expected to perform to the existing schedule. After all, if the schedule for this particular software isn’t met, the whole project will fall behind. Furthermore, the other team members are busy with their own work. Although they will answer an occasional question, they are quick to refer the new hire to documentation or Web resources. New employees on new projects don’t fare much better. Faced with a death-marchtype plan, the new employee is given the same workload as experienced employees. Furthermore, new-hire salaries are sufficiently high that experienced employees are not that sympathetic to the new hire’s plight, thinking to themselves, “When I started out,

Although formal education is crucial, a software engineer should also regularly (for example, every five years) prepare for and pass certification exams.

September/October 2002

IEEE SOFTWARE

39

The education received on best engineering practices and techniques to support various software lifecycle activities will benefit a job candidate for a lifetime.

I got a fifth of what these new hires are getting, so they should pull their own weight.” Whatever the reason, new graduates are expected to perform on the same level as their experienced counterparts. Everyone up the line is under schedule pressure, and the idea of an apprenticeship period is a foreign concept in software development. The best that the new employee can hope for is a sympathetic, experienced mentor who will coach him or her along. Prospective employees must look for those enlightened companies that can provide appropriate education and mentoring for their staff, and companies must recognize the need for apprenticeship and continuing education. The fact that books such as Death March and Slack exist and have a large audience suggests that this will not be an easy task. Myth 7 Software engineering programs will correspond to specific corporate requirements.

All you have to do is to look at current job postings to see a list of specific languages and tools, such as C, C++, Ada, UML, Visual Studio, XML, and ASP, along with the disclaimer that no experience is required. To quote from some recent newspaper classified ads: “Must know C/C++,” “Must be familiar with Accelerated SAP,” and “Must have experience with object-oriented programming in Java or C++.” Searching the job site www.monster. com, we found only one job description that talked about developing software requirements, designing, coding, and testing, along with using best engineering practices. We seem to be stuck in a time warp that emphasizes form over substance. There is no point in requiring experience in specific languages and tools—a software engineer can learn new languages and tools fairly readily, most of which will change in five years anyway. On the other hand, the education received on best engineering practices and techniques to support various software life-cycle activities will benefit a job candidate for a lifetime. It would seem that many employers are still looking for programmers who can produce code in specific languages using specific tools in the short term. They’re not looking for software engineers who can develop software using best engineering practices with a long-term 40

IEEE SOFTWARE

September/October 2002

view. This is because people in industry typically are looking for someone to start development work immediately—they do not have the time or interest to train their employees. Furthermore, companies expect their employees to leave in a year or two, so they assume they won’t benefit from the longer term SE knowledge that the employee might have. The events of recent years have supported this attitude; SE staff expect to change jobs regularly, in some cases for a healthy salary increase. So, we have a vicious cycle: Software staff change jobs regularly because companies make little investment in employee retention, and companies make little investment in employee retention because software staff change their jobs regularly. In fact, the company that invests in employee training might find that the employee adds the newly acquired skill to his or her resume to find a job. Another contributing factor is that industry managers typically started as programmers, with no SE background, so that is their frame of reference. They want someone like themselves when they were starting out. Companies also expect universities to use particular programming languages and tools in their curriculum, regardless of whether these languages and tools are the best vehicle to support the universities’ education goals. If you query a software executive on his or her needs and how universities can help to meet them, that executive’s first reaction is to list languages and tools. Only after some discussion does he or she move beyond this low-level litany to focus on the real education software engineers need.

H

ow should we, as SE practitioners and educators, respond to these myths? First, remember that the field is still young, so we can expect to see diverse opinions on various issues. Moreover, education is not a panacea. We are not going to cause SE to become a mature field overnight by fielding relatively small numbers of SE degree programs. We must foster stronger communication between diverse groups, such as various faculty groups, and between universities and industry. Myths tend to develop when there is little communication or when the communication that exists reflects our preconceived notions rather than objective assessment.

About the Authors

Universities and degree programs that have industry advisory boards report valuable exchanges of information through this mechanism. Faculty groups in different departments can also benefit from both informal and formal communications opportunities. We tend to lose sight of the fact that there might not be a right or wrong approach to SE education. There is ample opportunity to experiment. We do not need to pigeonhole SE education into one model or another just yet. If we experiment and track our results, we will learn what works over time. It is actually good to have many different kinds of degree programs because they will allow the proper environment for experimentation and discovery. So, maybe we just need to lighten up a little when we consider SE education degree programs, have fun developing and delivering these programs, and try to identify good educational models that work. At the same time, we need to figure out how to elicit industry feedback and incorporate it into degree programs in appropriate ways. Only then will SE reach the professional status it so richly deserves. References 1. W. Gibbs, “Software’s Chronic Crisis,” Scientific Am., vol. 271, no. 3, Sept. 1994, pp. 86–95. 2. H. Lieberman and C. Fry, “Will Software Ever Work?” Comm. ACM, vol. 44, no. 3, Mar 2001, pp. 122–124. 3. Computing Curricula 2001, ACM Special Interest Group on Computer Science Education, 2001, www.acm.org/sigs/sigcse/cc2001. 4. D. Bagert et al., Guidelines for Software Engineering Education, Version 1.0., tech. report CMU/SEI-99-TR032, Software Eng. Inst., Carnegie Mellon Univ., Pittsburgh, Pa., 1999. 5. T. Lethbridge, “What Knowledge Is Important to a Software Professional?” Computer, vol. 33, no. 5, May 2000, pp. 44–50. 6. ABET, ABET Criteria for Accrediting Computing Programs, 2002, www.abet.org/criteria.html. 7. D. Parnas, “Software Engineering Programs Are Not Computer Science Programs,” IEEE Software, vol. 16, no. 6, Nov./Dec. 1999, pp. 19–30. 8. M. Shaw, “Prospect for an Engineering Discipline of Software,” IEEE Software, vol. 7, no. 1, Jan./Feb. 1990, pp. 15–24. 9. M. McCracken, “Software Engineering Education: What Academia Can Do,” IEEE Software, vol. 14, no. 6, Nov./Dec. 1997, pp. 26–29. 10. M. Shaw, “Software Engineering Education: A Roadmap,” The Future of Software Engineering, A. Finkelstein, ed., ACM Press, New York, 2000, pp. 371–380. 11. S. McConnell and L. Tripp, “Professional Software Engineering: Fact or Fiction?” IEEE Software, vol. 16, no. 6, Nov./Dec. 1999, pp. 13–17. 12. E. Yourdon, Death March, Prentice Hall, Upper Saddle River, N.J., 1997, p. 218. 13. T. DeMarco, Slack, Broadway Books, New York, 2001, p. 226.

Hossein Saiedian is a professor of software engineering and an associate chair in the Department of Electrical Engineering and Computer Science at the University of Kansas. His primary research area is software engineering—in particular, models for quality software development. He is also interested in SE education and training and cochaired the ICSE’s Software Engineering Education track for 2000 and 2001 (and will cochair it again for 2003). He received his PhD in computer science from Kansas State University. He is a senior member of the IEEE and a member of the IEEE Computer Society and ACM. He is chair of the IEEE-CS TCSE’s Committee on Software Engineering Education. Contact him at the Dept. of EECS, Univ. of Kansas, Lawrence, KS, 66045; [email protected]. Donald J. Bagert is a professor of computer science and software engineering at the Rose-

Hulman Institute of Technology, where he is also the director of software engineering. His research interests include software process improvement, software tools for student advising, and software methodologies. He received a PhD in computer science from Texas A&M University. He is the steering committee chair for the IEEE Computer Society Conference on Software Engineering Education and Training, and the Professional Issues Editor for FASE, an electronic newsletter devoted to software engineering education, training, and professional issues. He is also a member of both the Educational Activities Board and the Professional Practices Committee for the IEEE Computer Society, and is a senior member of the IEEE. Contact him at the Dept. of Computer Science and Software Engineering, Campus Mail Box 97, Rose-Hulman Inst. Of Technology, 5500 Wabash Ave., Terre Haute, IN 47803; [email protected]. Nancy R. Mead is the team leader for the Survivable Systems Engineering team as well as a

senior member of the technical staff in the Networked Systems Survivability Program at the Software Engineering Institute. She is also a faculty member in the Master of Software Engineering and Master of Information Systems Management programs at Carnegie Mellon University. She received her PhD in mathematics from the Polytechnic Institute of New York, and a BA and an MS in mathematics from New York University. She is a senior member of the IEEE and IEEE Computer Society and is a member of the ACM. Contact her at the Software Engineering Inst., Carnegie Mellon Univ., Pittsburgh, PA 15213; [email protected].

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

Master software with these future topics: The Business of Software Engineering Model-Driven Development Managing Outsourced Projects Software Geriatrics

IEEE

Visit us on the web at

h t t p : / / c o m p u t e r. o r g / s o f t w a r e September/October 2002

IEEE SOFTWARE

41

focus

educating software professionals

Teaching PSP: Challenges and Lessons Learned Jürgen Börstler, Umeå University, Sweden David Carrington, University of Queensland, Australia Gregory W. Hislop, Drexel University Susan Lisack, Purdue University Keith Olson, Utah Valley State College Laurie Williams, North Carolina State University

atts Humphrey from the Software Engineering Institute at Carnegie Mellon University developed the Personal Software Process and first taught it as a graduate course at CMU in 1994. He says its goal is “to help you be a better software engineer ... As you study and use [PSP’s techniques], you will soon know how

W Software engineering educators must provide educational environments where their students can learn about the size and complexity of modern software systems and the techniques available for managing the difficulties associated with them. 42

IEEE SOFTWARE

to define, measure, and analyze your own processes.”1 The PSP adapts a continuous improvement model to the specific needs of an individual software developer who wants to be more productive and produce higher quality software. In particular, the PSP targets the process used to individually design and develop software and incorporates ways to measure and change the process to achieve higher quality and increased efficiency. Humphrey designed the complete PSP process as a semester-long university course for graduate students.1 A student or professional learning to integrate the PSP into his or her process begins at Level 0 and progresses in process maturity to Level 3 (see Figure 1). During the course, each student accumulates personal historical data to create estimates and measure improvement. Each level incorporates new skills and techniques to improve software quality into the student’s process

September/October 2002

and has detailed scripts, checklists, and templates to guide the student through the required steps. The measurement-based feedback in the PSP helps each student improve his or her own personal software process. Thus, Humphrey encourages customization of these scripts and templates as the students use this feedback to understand their own strengths and weaknesses. Students generally observe significant quality improvements as they progress through the four levels.2 Why teach the PSP? The concepts that a graduating software engineer should know are well defined in the Stoneman version of the Guide to the Software Engineering Body of Knowledge. 3 Of the 10 areas defined in SWEBOK, the PSP covers five: design, construction, testing, process, and quality. During a PSP course, students receive concrete experience in soft0740-7459/02/$17.00 © 2002 IEEE

PSP3 Apply PSP2 iteratively

ware metrics, life cycles, quality, and process improvement. The PSP lets students cover these topics both in theory and in practice. With the increasing complexity of software projects, a growing emphasis exists on process maturity as a means of providing a quality product when time and budget constraints arise. The PSP provides a framework for high maturity processes scaled for an individual software engineer. Hence, it provides a meaningful way to instill process awareness in software engineers. Even if students don’t use the PSP again, improving and making them aware of their programming habits will help them in their future academic and professional careers. With the recent rise in popularity of agile software development methodologies4 (such as Extreme Programming), some educators might question the need to teach the PSP. Agile methods are highly incremental approaches that de-emphasize front-end analysis, design, and documentation in favor of communication and tacit knowledge transfer through human communication. However, Barry Boehm believes that both plan-driven methodologies (such as the PSP) and agile methodologies are important. “Both agile and plan-driven methods have a home ground of project characteristics in which each clearly works best and where the other will have difficulties. Hybrid approaches that combine both methods are feasible and necessary for projects that combine a mix of agile and plandriven home ground characteristics.”5 Teaching options Depending on the environment, many ways to teach the PSP exist. The PSP material is quite extensive, so instructors might need to tailor and customize it to meet the needs of their class. We have identified three primary factors that influence the teaching of the PSP: course environment, coverage level, and support tools. A PSP course’s environment depends on the target audience, course level, and subject content. For professional software developers, teachers generally present the PSP in a distinct course using Humphrey’s A Discipline for Software Engineering.1 For students in computing programs, educators have tried integrating the PSP into almost every type of course from first to fourth year—as well as at the graduate level. Pub-

lished case studies show that teaching the PSP can PSP2 Manage product quality be successful in any environment, but they also mention the difficult intePSP1 gration with first (CS1) Estimate using historical data and second semester (CS2) computing courses. Those who do integrate PSP0 Gather data on current process the PSP into CS1 often use Humphrey’s An Introduction to the Personal Software Process.6 Figure 1. The levels This shorter, easy-to-read of the Personal book is specifically intended for teaching Software Process. first-year students the basic principles of using disciplined software processes to produce high-quality software. In this article, we refer to this text as PSP-lite. Another factor to consider when teaching the PSP is coverage of the material. In dedicated PSP courses, teachers introduce its four levels using 10 exercises that A Discipline for Software Engineering describes.1 However, when integrating the PSP with other course topics, such a complete introduction is not feasible. Humphrey recognizes this problem and gives some suggestions for variations.1 However, all the proposed variations are still staged introductions involving at least seven assignments. Instead, some teachers introduce just one specific (often slightly modified) PSP level and use it throughout the course, making it possible to reduce the number of exercises. The PSP can also be integrated into CS1 or CS2 using PSP-lite. Having the proper tools to support the tasks involved in PSP related activities is also important. Manual data collection often leads to incomplete, inconsistent, or erroneous data sets. Many PSP studies mention this observation and identify tool support as an important issue. The instructor’s guide accompanying A Discipline for Software Engineering provides spreadsheets to calculate statistics.1 However, Philip Johnson and Anne Disney identify the main problem as student data collection, not teacher data collection and manipulation.7 University experiences Using the three factors, environment, coverage, and tool support, Table 1 summarizes our experiences teaching some PSP variations at five universities. September/October 2002

IEEE SOFTWARE

43

Table 1 PSP results at five different universities University

Environment

Umeå University

2nd-year C++ course 2nd/3rd-year SE

University of Utah

CS1/CS2

Montana Tech

Drexel University

Tool support

Comments

Locally developed tool

Optional usage was ineffective Team project (developing a PSP tool); good learning experience Well received without much burden on students or teaching staff Students in pairs outperformed students working individually Students felt the data collection for the PSP was too much for an introductory programming course

Modified PSP1 only PSP-lite across two courses Full PSP

Senior SE Purdue University

Coverage

Locally developed tool

CS1 only, CS1/CS2, sophomores CS1

PSP-lite

Junior CS/SE

Full PSP

Used provided spreadsheets

Graduate SE process course

Full PSP through PSP2

Standard forms

Used provided spreadsheets PSP-lite

Umeå University Umeå University used the PSP to exemplify disciplined software development. To minimize changes to the curriculum, we integrated certain basic PSP topics into two existing courses: a second-year C++ course and a second- or third-year software engineering course. For the C++ course, we developed an extra-lite version of the PSP to deliver an essential PSP lesson—namely, “plan, track, and review what you did.” We introduced this adaptation of PSP in two extra 45-minute lectures and used it throughout all the exercises. We felt three medium-size exercises, with one or two weeks to complete each, were sufficient to collect historical data for trend analysis. We also provided a tool to support data collection and minimize data errors. PSP use was optional—only six of the 78 students used it throughout the course. None of these six students reported any perceived process improvements during the course. For the remaining students, the main reason for abandoning the PSP was the feeling that it would impose an excessively strict process on them and that the extra work would not pay off.8 Similar experiences occurred in other courses where PSP usage was optional. As a result, we do not advise optional PSP use. In the software engineering course, we tried a different approach to familiarize students with the PSP. This course combines a group project and a theory track with lectures and assignments. During the theory track, we introduced the PSP as a bottom-up 44

IEEE SOFTWARE

September/October 2002

Taught as an add-on to CS1, exercises not correlated; Marginal success Students resisted data collection at first, but were very positive about the results at end of term Very good learning experience although somewhat labor intensive for instructor and students

approach to control software development, using published case studies extensively to show students that it was successful. During the group project, students developed tools to support data collection and trend analysis for the PSP. This forced students to actively acquire information about the PSP and its usage. Examination results showed that this teaching method helped the students understand the problems that PSP can solve. University of Utah For many years, teachers at the University of Utah used PSP-lite to teach the PSP to freshmen, integrating the material into both CS1 and CS2 and giving approximately five half-hour lectures on the material in each class. CS1 covers PSP-lite’s second half, which deals with defects (recording, reduction, prevention, and design–code reviews). Students estimate, track, and analyze the defects they remove from their programs. The book’s first half, which focuses on resource estimation and tracking, is taught in CS2, where the students estimate, track, and analyze the time they spend on their programs while continuing defect tracking and analysis. Teachers provide students with reports on class statistics and ask them detailed questions about their own PSP data on exams. Neither the teaching staff nor the students found integrating this material across the two classes to be difficult. Students reported that the knowledge gained on software engineering principles greatly aided them in obtaining summer internships.

Teachers taught the full PSP as well as a customized form of it that included pair programming during the last offering of a senior software engineering class in 2000. The university moved the software engineering class to the sophomore year and now relies on the PSP-lite taught in the freshman year. The class was a formal experiment to analyze the professed benefits of pair programming; students experienced even greater success when practicing pair programming while following PSP’s practices.9 These students passed 15 percent more black-box test cases and spent approximately half the elapsed time when compared with students who worked alone following the PSP. Additionally, the pairs enjoyed themselves more, had higher confidence in their work, and encouraged each other to follow PSP practices. Purdue University Purdue University’s Department of Computer Technology incorporated PSP materials into its introductory programming courses on two different occasions: once in the first course only and once across the two-semester sequence of introductory programming courses. Students in these courses were primarily sophomore computer information system majors who had already taken several courses to develop their computer literacy. Faculty integrated the PSP-lite book and activities into the existing programming courses by covering half the book in each course. Students collected the PSP data as part of each weekly programming assignment. Because other topics needed to be covered, teachers devoted a limited amount of lecture time to discussing the PSP materials. A course instructor checked the PSP data occasionally to make sure students were completing the activities correctly, and students completed a brief questionnaire at the course’s end to rate their attitude toward the PSP. In general, the students viewed PSP activities as extra work added to the regular programming assignments and did not appreciate the potential benefits of a disciplined process. Susan Lisack reported that students made numerous errors on the PSP forms, and expressed negative attitudes toward the PSP on the course exit survey.10 In short, students felt they were already busy enough learning new language syntax and

the program development environment. However, the students offered several suggestions to improve future course offerings. More automated tools would make it easier to record the required data, especially when more than one form needed the same data. The instructor should review student data early and provide immediate feedback about recording errors so that the collected data is meaningful. The instructor should also present actual data from the class to illustrate the PSP’s benefits. Most importantly, students strongly recommended placing the PSP topics in a later programming course. Faculty responded by placing PSP topics into a graduate-level course on software processes and possibly including them in an upper-level undergraduate software methodologies course.

The most common reaction was the students feeling more aware of their programming practices and shortcomings.

Montana Tech of the University of Montana We also taught the PSP-lite and PSP in courses at Montana Tech. Our best success was with the full PSP, taught at the junior level. All students had completed two courses in data structures and algorithms and had a high level of competency in programming. The course met three days per week, and used the A series of exercises in Humphrey’s text.1 Student reaction was initially resistive, but in the end, the course received excellent student evaluations. The most common reaction was the students feeling more aware of their programming practices and shortcomings after the course. Faculty did not attempt to measure the students’ performance in subsequent courses, but in senior exit interviews, the students all stated that the course made them more effective programmers. Drexel University The PSP continues to provide the focus for a graduate course in process improvement offered at Drexel University once or twice each year since 1996. The students in the course are masters students in both information systems and software engineering. The course addresses the PSP as presented in A Discipline for Software Engineering.1 Recently, the course has also included some coverage of agile methodologies as a point of contrast.4 The course includes only seven PSP exercises because Drexel operates on a quarter term with 10 weeks of classes. This is sufficient for students to try the PSP through Level September/October 2002

IEEE SOFTWARE

45

While understanding the PSP’s mechanics is relatively easy, developing an appreciation for its goals and potential is much harder.

2. Drexel faculty has used both the A and B series of exercises, and has found that the B series works much better for the information systems students.1 Teachers use the standard PSP forms and provide students with some example spreadsheets to demonstrate various PSP calculations such as regressions. Both instructors and students find the course work labor-intensive, but manageable. Students in this course have a wide range of experience and programming ability. They include people with only minimal classroom programming experience, software developers with extensive experience, and managers who have not programmed in many years. These varying perspectives help enrich the discussion but also require the instructor to deal with each group’s different problems approaching the PSP. Overall results in teaching the PSP have been excellent. The faculty members teaching the course find it effective as a teaching vehicle. Some students are initially resistant to the PSP’s requirements, and instructors have to be prepared to deal with these objections. However, by the course’s end, students generally report that they found the experience valuable, and several of them report taking at least some PSP parts back to their work environments. Challenges for students The PSP is less about mechanics and more about instilling good habits and professional attitudes. As such, learners should apply PSP practices as a regular part of their studies, not just in a single course. Although widespread support is preferred, few universities are able to provide reinforcement throughout the curriculum because broad faculty support is required. While understanding the PSP’s mechanics is relatively easy, developing an appreciation for its goals and potential is much harder. For students without industrial experience, understanding the problems of large-scale, team-based software development is difficult. Student programs tend to be small and short-lived. Students experience neither the problems of post-release defects nor the value of high-quality software in production operation. Students with industrial experience can have different problems. They usually

46

IEEE SOFTWARE

September/October 2002

have first-hand experience with the key problems the PSP addresses. They understand why solving these problems would be good, but often they do not see the PSP as a workable solution. Many experienced students have ingrained programming habits and imagine that the PSP adds administrative overhead. Until they see the benefit for themselves, they can be resistant to it. In addition, these students might believe that a different way of working is impossible because managers will not support PSP practices. Experienced or inexperienced students could also miss the PSP’s value if they are unwilling to make a serious effort to try it. Students who simply do the minimum amount of work to get through the course are unlikely to gain any appreciation of the potential value that the PSP has for them personally as working software engineers. Many universities teach software engineering practices that industry does not widely practice. This presents a particular problem for students when they move into the workplace. The typical situation is that a student joins a software development organization that neither knows about nor accepts and applies practices such as the PSP. Most students in this situation are likely to mold themselves to the local culture rather than using practices that their peers and managers might not respect. Challenges and suggestions for teachers With the rapid advances in computing technology, fitting the PSP into an already crowded curriculum is a continual problem for teachers. Universities are under increasing pressure to produce students with in-depth knowledge of the latest technologies for an industry with a chronic shortage of technical people. The mechanics of the introductory PSP are well within the reach of most students, although there is a need to devote teaching time to the PSP. The requirement for application of basic regression and correlation makes the full PSP difficult for first year undergraduates. However, it is reasonable to teach basic PSP processes in the early years, and then address topics requiring statistics in the later years.

Based on our experiences, we offer some guidance and advice to teachers interested in introducing the PSP. Student motivation Motivating students about the PSP’s benefits is essential. Students enter a university with a predisposition toward coding without designing, and they do not see the data collection required for the PSP as enjoyable or useful. Teachers must overcome students’ limited knowledge of commercial-scale software development and the work environment, and provide them with an appreciation for PSP’s data collection, analysis, and measurementbased feedback. Introducing the PSP early may help students form good software habits, but teaching programming and the PSP simultaneously might cause cognitive overload for students. The first half of PSP-lite deals with time recording, effort estimation, and commitment making, while the second half deals with defect reduction, prevention, and recording. Through the eyes of a beginning software engineering student, the material on defects in the book’s second half seems more relevant. Even after just one program, many students have experienced the frustration of losing several hours to a small syntax or semantic error. They are motivated to reduce the frustration of defect elimination and are receptive to learning how to control their defects and minimize the amount of time spent debugging. We suggest that educators teach the material on defects prior to the material involving time recording. Students are also more motivated to learn the PSP’s lessons if the instructor can share stories of how previous students fared much better in job interviews because of their PSP background. Employers are impressed with students who can intelligently discuss the PSP and the principles it emphasizes. Integration and adaptation We have seen greater long-term success instilling the lessons of high-quality software development when teachers integrate the material across the curriculum.11 Integration generally requires a commitment from multiple faculty members. It is not essential that these other faculty members inspect the students’ time and defect logs, but these instructors can support the PSP’s les-

sons by asking students to estimate, track, and report their PSP data. A teacher’s ability to adapt the PSP to the needs of their students is also important. Although it makes sense for experienced software engineers to develop the early PSP exercises in a waterfall-style process, this is normally not true for undergraduates who lack the programming experience to design and code even a small program in one iteration. Early on, teachers should show such students how to plan and perform multiple iterations. However, students should be discouraged from the practice of design-codecompile-test cycles at the single statement level. Apart from the practical problems of recording process data for such fine-grained iterations, such practices do not scale up to industrial-scale software development. Dealing with data The PSP lets students collect their own data to show how they’ve improved as they use better software engineering practices. When teachers reduce the number of exercises, students cannot see this progress as clearly because of a lack of data. Using published PSP data from other PSP users to talk about the PSP is possible, but it rarely has the same impact. The power of the PSP is when students realize that it works for them by seeing their own work improve. Teachers must be able to process and present class data in a meaningful way. Students appreciate seeing aggregate class results on time estimation versus actual time spent (minimum, maximum, and average). They also like to see class quality trends (defects per KLOC) as they learn and apply new programming and quality techniques. To satisfy this desire for feedback, teachers need mechanisms for reliably and efficiently collecting PSP data from students and converting it into presentations. Ideally, these mechanisms would also let teachers provide individual feedback where appropriate and check the authenticity and validity of student data. Several tools are now available to support student data collection and analysis—for example, LEAP (http://csdl.ics. hawaii.edu/Tools/LEAP/LEAP.html) and the Process Dashboard (http://processdash. sourceforge.net). The challenge is to have tools that are convenient to use and don’t distract from the work itself. September/October 2002

IEEE SOFTWARE

47

Teachers must also be conscious of the potential for misuse of PSP data. If students ever believe that this data is being used for grading purposes, they are likely to manipulate the values in an attempt to gain better grades. Students starting to use PSP often feel that there are ideal or target values that they should aim for. Teachers need to make it clear that the activities of collecting and analyzing the data are important, not the particular values.

1, Mar. 2001, pp. 7–20. 10. S. Lisack, “The Personal Software Process in the Classroom: Student Reactions,” Proc. 13th Conf. Software Eng., Education, and Training, IEEE CS Press, Los Alamitos, Calif., 2000, pp. 169–175. 11. M. Towhidnejad and T. Hilburn, “Integrating the Personal Software Process (PSP) across the Undergraduate Curriculum,” Proc. 27th Frontiers in Education Conf., IEEE Press, Piscataway, N.J., 1997, pp. 162–168.

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

About the Authors

W

e hope our experiences and experiments teaching the PSP provide guidance and advice for others interested in introducing the PSP into their software engineering courses. We continue to use elements of the PSP in our teaching because we believe that it is one of the few coherent and explicit approaches for teaching students about effective software engineering practices for developing high quality software. As with the PSP itself, the quantitative feedback available from teaching the PSP allows teachers to base their improvements on data, not just perceptions.

Acknowledgments This article is an outcome of the presentations and discussions presented at a workshop on teaching the PSP in universities held 20 February 2001 at the Conference on Software Engineering Education and Training (CSEE&T) in Charlotte, North Carolina (see www.spsu.edu/oce/cseet2001/registration.htm).

1. W.S. Humphrey, A Discipline for Software Engineering, Addison-Wesley, Boston, 1995. 2. A. Wesslén, “A Replicated Empirical Study of the Impact of the Methods in the PSP on Individual Engineers,” Empirical Software Eng., vol. 5, no. 2, June 2000, pp. 93–123. 3 A. Abran et al., Guide to the Software Engineering Body of Knowledge: Trial Version, IEEE CS Press, Los Alamitos, Calif., 2001. 4. A. Cockburn, Agile Software Development, AddisonWesley, Boston, 2002. 5. B. Boehm, “Get Ready for Agile Methods, with Care,” Computer, vol. 35, no. 1, Jan. 2002, pp. 64–69. 6. W.S. Humphrey, An Introduction to the Personal Software Process, Addison-Wesley, Boston, 1997. 7. P. Johnson and A. Disney, “A Critical Analysis of PSP Data Quality: Results from a Case Study,” Empirical Software Eng., vol. 4, no. 4, Dec. 1999, pp. 317–349. 8. S. Olofsson, Evaluation of the PSP in the Undergraduate Education, tech. report UMNAD 272.99, Dept. of Computing Science, Umeå Univ., Sweden, 1999. 9. L. Williams and R. Kessler, “Experimenting with Industry’s Pair Programming Model in the Computer Science Classroom,” Computer Science Education, vol. 11, no. IEEE SOFTWARE

he leads the software engineering and computer science education groups. Contact him at Department of Computing Science, Umeå University, SE-901 87 Umeå, Sweden; jubo@ cs.umu.se.

David Carrington is a reader in the School of Information Technology and Electrical

Engineering at the University of Queensland, Australia, where he is the program director for the software engineering program. His research interests include software engineering processes, methods and tools, and software engineering education. Carrington is also a visiting scientist with the Software Engineering Institute at Carnegie Mellon University. Contact him at [email protected].

Gregory W. Hislop is an associate dean and faculty member of the College of Information Science and Technology at Drexel University, where he coordinates the college’s software engineering and information systems programs. He has nearly 20 years industrial experience in software engineering and enterprise systems management. Contact him at [email protected].

Susan Lisack is an assistant professor in the Computer Technology Department at Pur-

due University, with interests in the areas of programming and databases. She holds certification as an Oracle8i Certified Database Administrator and also coordinates the departmental cooperative education program. Contact her at [email protected].

References

48

Jürgen Börstler is an associate professor of computer science at Umeå University, where

September/October 2002

Keith Olson teaches at Utah Valley State College in Orem, Utah, where he coordinates the software engineering program for the Department of Computing and Networking Sciences. His principal research interests are in software processes and estimation. Contact him at [email protected].

Laurie Williams is an assistant professor of computer science at North Carolina State

University. She received her undergraduate degree in industrial engineering from Lehigh University. She also received an MBA from Duke University and a PhD in computer science from the University of Utah. Her research interests include software development process, software testing and reliability, software security, and Ecommerce. Contact her at [email protected].

focus

educating software professionals

Simulating a Software Engineering Apprenticeship Ken Surendran and Helen Hays, Southeast Missouri State University Andrew Macfarlane, QCOM

lthough some professions require a period of internship, residency, or apprenticeship before marking an individual as a qualified member, software engineering does not. Universities have started offering curricula in SE, and many continue to offer SE as a set of courses in their computer science curricula. Because the global demand for software engineers far exceeds the supply of SE graduates, CS and applied CS graduates frequently fill this gap. So, to enrich SE courses in the

A A framework for a software engineering apprenticeship and an enriched model for SE professional development will help establish steps for attaining software engineering professional status. The current practice of industrysponsored projects in coursework and internships is insufficient.

0740-7459/02/$17.00 © 2002 IEEE

CS and applied CS curricula with opportunities to practice SE concepts and principles, we support an SE apprenticeship, simulated through system development projects. In this article, we examine the prevailing SE body of knowledge and perspectives on SE education to construct a model comprised of knowledge areas for SE professional development. We use this model and Benjamin Bloom’s taxonomy1 to delineate knowledge areas that an apprenticeship could best handle. We then establish a framework for SE apprenticeship, which we use to examine practices for apprenticeship simulations in three academic institutions that offer CS, applied CS, or computing systems curricula. Body of knowledge What knowledge does a software engineer require for professional development? We addressed this question by constructing an

SE professional development model, which classifies knowledge areas into four categories: core, support, foundation, and context (see Figure 1). This type of multitiered architecture can help model the distribution of SE professional development responsibilities between academia and industry. Knowledge areas We used several KA sources in our development model. The Software Engineering Body of Knowledge, prototyped in the technical report by Thomas Hilburn and his colleagues2 and detailed in the Guide to the Software Engineering Body of Knowledge (SWEBOK),3 contains 10 software KAs: requirements, design, construction, testing, maintenance, configuration management, management, process, tools and methods, and quality. Each of these have numerous topics and subtopics. September/October 2002

IEEE SOFTWARE

49

A difficult thing to achieve in a curriculum is realism—real products signifying tangible, relevant achievements and real people signifying collaborative effort.

Hilburn and SWEBOK also list KAs from related disciplines: CS, mathematics, project management, computer engineering, systems engineering, management and management science, and cognitive sciences and human factors. Bertrand Meyer lists goals for a software curriculum:4 ■ ■ ■

■ ■

Principles—recurring concepts such as recursion and debugging Practices—practical techniques such as user interfaces and metrics Applications—traditional software domains such as operating systems and compilers Tools—such as industrial strength programming languages Mathematics

Barry Boehm and his colleagues list, aside from CS, three other underlying sciences for SE: domain science, behavioral sciences, and economics.5 Development model In Figure 1, the core category comprises the first five traditional waterfall model KAs from SWEBOK. The support category includes SWEBOK’s configuration, quality, and methods and tools KAs and two topics from its software engineering management KA: project management and metrics. We consider formal methods part of the methods and tools KA. The foundation category contains underlying KAs from CS, mathematics, and traditional generic software domains.4 The context category includes four components drawn primarily from Boehm and his colleagues:5 ■

■

■

■

50

IEEE SOFTWARE

SE process deals with organizational, team, and individual development (including SWEBOK’s process KA). Practice deals with producing and operating software products under society’s prevailing legal and ethical norms. Economics concerns the software products’ value realization in a business context. Scope comprises the application domain context from which the inspiration for new and refined software products emerges.

September/October 2002

Two types of application domains exist: generic software (such as database, artificial intelligence, compiler, and operating systems) and field-specific software (such as business information, bioinformatics, and entertainment), which can heavily depend on generic software. We include the generic software application domains in the foundation category and the field-specific ones in the context category. Knowledge elements The idea of a software engineering apprenticeship (SEA) is not new. For instance, James Bach and W. Michael McCracken discuss apprenticeship and what an academic education can and should offer future SE practitioners.6 However, considerable diversity exists among opportunities for practical experience at universities, and not many universities have a formal SEA component built into their curricula. As we propose a standard SEA framework, we examine the model in Figure 1 to identify appropriate knowledge and the mastery levels that we expect students to achieve. We then define the mastery levels in terms of Bloom’s taxonomy, which helps classify cognitive educational goals. We list the progressive mastery levels of Bloom’s taxonomy and expected behaviors: ■ ■ ■

■

■ ■

Knowledge requires recognizing or recalling ideas Comprehension requires rearranging, interpreting, and extrapolating ideas Application requires restructuring a problem and selecting and using appropriate ideas to solve the problem Analysis requires breaking down material and classifying its elements, identifying relationships between elements, and recognizing the whole’s organizational structure Synthesis requires combining elements into a new structure Evaluation requires making criteriabased judgments about the works’ internal quality and usefulness for achieving desired ends

The KA experts who contributed to SWEBOK provided mappings from eight of the KAs and associated topics to the six levels of Bloom’s taxonomy, indicating for each topic the mastery level expected of a graduate with

Context category

SE process SE practice four years’ SE experience. (Professional development context) (Society context) (Ratings are omitted for the construction and maintenance KAs.) The SWEBOK SE project management Support SE methods and tools category mappings generally classify Software configuration management processes and concepts Software quality management (such as the requirements SE metrics engineering process, general Core category design, and SE process) at Software Software Software Software Software the knowledge and comprerequirements design construction testing maintenance hension levels. This is conMathematical foundations, sistent with the progressive Foundation algorithms and data structures, category nature of Bloom’s taxoncomputer architecture, omy, in which the higher programming language, generic software levels build on and incorpoSE economics SE scope rate lower levels. SWEBOK (Business context) (Application domain context) generally places activities performed by analysts, programmers, and low-level managers (such as software design notations, test levels, and personnel manageFigure 1. A software ment) at the application or analysis level. Ac- an academic program, but depth is acquired engineering (SE) tivities performed by quality assurance spe- from experience. professional cialists and higher-level managers (such as development model. common planning activities and measurement Framework applied to quality assurance) are generally An apprenticeship program extends the placed at the synthesis or evaluation level. academic curriculum. A difficult thing To develop a framework for the SEA, we to achieve in a curriculum is realism6— must determine what mastery levels stu- real products signifying tangible, relevant dents can attain through coursework in the achievements and real people signifying CS-SE curriculum, in the SEA, or during the collaborative effort. So, an apprenticeship first four years of work experience after program tries to provide opportunities to start practicing the profession under supergraduation. Generally, students will achieve knowl- vision and guidance in an industrial setting. In the ideal SEA, a mentor in the industry edge and comprehension levels through coursework. This includes the foundation plays a critical role in guiding the apprencategory in Figure 1 and underlying methods tice’s professional development by assigning different area supervisors, each charged with and concepts for the remaining categories. The SEA will concentrate mainly on analy- giving the apprentice specific tasks and sis and application levels for the core and sup- closely monitoring progress in that area. The port categories in Figure 1. The apprenticeship SEA should emphasize honing skills to will also require comprehension of a particu- achieve application and analysis mastery levlar application domain (“SE scope” within the els in the core and support category KAs context category). To some extent, we can ap- (Figure 1). The apprenticeship should also let ply concepts from other components of the the individual explore various possible roles context category in the SEA: process (the Per- to help plan a professional specialization. sonal Software Process), economics (feasibility Furthermore, mechanisms must be built into the program to measure accomplishments. studies), and practice (ethics). The SE professional will acquire higher These elements lead to a three-dimensional mastery levels through work experience framework for an SEA (see Figure 2) with with large-scale, real-world problems after practice, role, and evaluation as its axes. graduation. As Bach and McCracken point out, prior to graduation, students are inten- Practice The practice axis addresses mastery of tionally exposed to a broad set of topics in

September/October 2002

IEEE SOFTWARE

51

Figure 2. The framework for a software engineering apprenticeship.

Evaluation

Overall performance evaluation by mentor Quality management Performance analysis by supervisor Internal customer feedback Planning process

Individual

System artifacts

Organizational

Progress reports Journal writing Activity log Maintain Programmer Implement Designer Design Analyst Analysis Project Assistant Test Review Role

the core and support categories in Figure 1. We apply a chronological order often used for new employees, in which they are first exposed to best practices in legacy systems by working on maintenance projects and gradually progress through those areas that require analysis mastery. (We suggest this order for the apprenticeship, not for the initial introduction of concepts in coursework.) One of the authors, while serving in a company as a mentor for summer interns, tried the suggested development order and found it effective for training purposes, with the following advantages: Starting with maintenance tasks (involving coding) in a system with well-written documentation and high-quality implementation will make the task easier. Additionally, this order stresses the efficacy of good SE practice and sets the tone for the quality of work expected when the apprentice moves on to developing a new system. Because undetected errors in the earlier stages of the software development life cycle are more costly to correct than those discovered later, the reverse-order exposure should minimize risks when using novices. So, acquiring implementation skills takes precedence over design, and design over analysis. The apprentice program has modest intentions and does not aim to develop the apprentice into a systems architect by the pro52

IEEE SOFTWARE

September/October 2002

Practice

gram’s end. However, the apprentice should get opportunities to examine system architectures that are already in place (to understand the rationale behind the choices made) and to apply the concept of reuse (to use frameworks). The program should expose the apprentice to at least one field-specific application domain during analysis. Testing requires experience in all core skills to carry out the integration and system-level tests. Testing here should include preparing specifications and test cases, as well as using those prepared by others. Getting involved in the various review activities in product development is important. Because review is an evaluative process, the program addresses it at the end. Role Items on the role axis parallel the practice axis and indicate the roles the apprentice will fill when performing the various practice items. Whereas practice helps hone the hard skills needed in the core and support categories, the roles (which involve professional relationships) provide opportunities to develop soft skills (also called value skills7). We use common names for roles and order them to match with the practice elements. Soft skills requirements also increase as the apprentice progresses from programmer to analyst. When you discover an ap-

prentice with higher potential, you could give additional higher-level responsibilities (such as active participation in review processes) to develop integrative and evaluative skills. The apprentice should work on different projects (or at least different subsystems of a project) under different roles. As a project assistant, the apprentice can get an integrated view of product development activities (especially supportive ones) and can start appreciating the importance of delegation, value of metrics, and rationale for business-driven decision making. Evaluation On the evaluation axis, we suggest two types of mechanisms: individual and organizational. On the left side of the axis (see Figure 2), individual evaluation components include PSP (Personal Software Process)8 mechanisms for professional development. We consider two of these—planning and quality—essential because they help develop estimation and reviewing skills. Keeping a personal journal helps the apprentice note lessons learned and important observations pertaining to ethics, quality, standards, and metrics. On the right side of the axis, the organizational mechanisms are for formal evaluation of the apprentice’s performance. Although the overall performance evaluation lies with the mentor, a portfolio of evidence is built over the apprenticeship. This evidence consists of various supervisors’ observations and assessments, which are, in turn, based on feedback from team members and internal customers and on the system artifacts the apprentice produced. The program logs all activities to assess productivity determination, quality awareness, and standards adherence. The weekly progress report lists accomplishments, lapses, and plans of remedial actions in a formal document. The apprentice consolidates these progress reports into a final report containing a list of achievements, improvement in work quality, familiarity with metrics, observations pertaining to ethical and legal issues, and identified areas for further improvement. SEA criteria An ideal apprenticeship program would require an additional year of practicum following the usual CS-SE four-year program. This would help instill best practices in the

application of KAs in the core and support categories for realistic product development, in the application of instruments for evaluating and managing professional development, and in the delivery of value-rich, quality products. You could use the SEA criteria in Figure 3 to measure the value of an apprenticeship program. Simulation Using the assessment criteria in Figure 3, we examine three academic programs that offer project-based SE courses. Additionally, we suggest a coordinated internship program. Three programs The first program (Prog-1), at a Midwestern US state university, has a two-semester sequence of SE courses. The first course emphasizes systems analysis and design, and students work in teams on a project provided by the instructor. Both CS and MIS students attend this course, and teams contain a mix of students from the two programs. The second course emphasizes the Rational Unified Process and project management, and student teams (of five to six members) apply RUP to develop a clientsponsored project. Some clients are internal to the university; for example, one team, working for a professor of communication disorders, developed an expert system for diagnosing and recommending treatment for speech articulation disorders. Other clients are from business and industry. For example, one team developed and installed a purchase order management system for a small local industry that manufactures radiators. The second program (Prog-2), at an institution of technology in New Zealand, has a two-semester sequence of systems analysis and design courses and a project management course, followed by a double-credit, one-semester capstone project course. In the capstone course, students work in teams of two to find (with the assistance of faculty or an advisory board), propose, and develop a client-sponsored project, including all standard system artifacts. For example, one team worked closely with a software house to produce a Web-based change management system. The third program (Prog-3),9 at a Midwestern US institution of technology, has a two-quarter SE course followed by a two-

An ideal apprenticeship program would require an additional year of practicum following the usual CS-SE four-year program.

September/October 2002

IEEE SOFTWARE

53

1. Assures application and analysis mastery of core and support knowledge areas (KAs): 1.1 Exposes the apprentice to the best products (all system artifacts) 1.2 Requires the apprentice to use best practices in product development 1.3 Involves the apprentice in dealing with internal and external customers (information gathering, documentation, and presentations) 1.4 Assigns quality assurance tasks to the apprentice (test specification, test case preparation, and test log) 1.5 Involves the apprentice in formal review processes in product development 1.6 Exposes the apprentice to scaling up from smaller- to larger-sized projects 2.

Assures application mastery of context KAs: 2.1 Coaches the apprentice in the development of interpersonal skills 2.2 Involves the apprentice in value judgment exercises (economic feasibility) 2.3 Exposes the apprentice to processes and tools (such as the CMM and ISO) used for continuous quality improvement 2.4 Measures apprentice performance periodically (strengths, weaknesses, and plans for improvement)

Figure 3. The criteria for an ideal software engineering apprenticeship.

quarter capstone system development project. In the SE courses, students work on client-sponsored projects and generally produce prototype systems. For example, the State Supreme Court sponsored a project for administering continuing education for registered lawyers. For the capstone project, students work in teams of four to five members on a project that an external client must sponsor. For example, one team, working for a pharmaceutical company, developed a scheduling system for manufacturing antibiotics and integrated it into the company’s spreadsheet system. Another team, working for a printer manufacturer, produced a prototype 3D workbench program for designing an optical system of lenses and mirrors. Simulation evaluation Table 1 evaluates the three programs using the SEA criteria in Figure 3. The first two programs provide a safer approach in that the client-sponsored project is assigned for the capstone course after the students gain experience with an incourse project. However, the third program (Prog-3) offers the greatest potential for simulating the SEA because it uses clientsponsored projects both for in-course and capstone purposes. Also, due to the university’s location and nature, a wider variety of projects from large companies is available. Although such client-sponsored projects can simulate quite a few SEA criteria, they lack the depth required in dealing with realworld, large-scale projects. For example, they do not provide opportunities to work on legacy systems (Criterion 1.1), and individuals can assume only a few critical roles required in the core category. Although a few

54

IEEE SOFTWARE

September/October 2002

curricula10 are built around the PSP and seem to address individuals’ quality improvement (Criterion 2.3), simple client-sponsored projects do not simulate them. Moreover, these projects totally fail to simulate some of the criteria pertaining to the context category KAs (particularly, the process and practice) and partially fail to simulate a few in the support category (metrics and quality). Students will get only limited experience in quality assurance (Criterion 1.4) because only a few of them will perform the integration and system testing. In the project courses, most of the feedback is for the whole group, so the individual doesn’t get a detailed appraisal, and Criterion 2.4 is not met. Coordinated internships and co-ops Project-based courses are a necessary component of the SE curriculum, but they do not provide the full apprenticeship experience needed to prepare highly qualified SE professionals. An additional experience that you can provide during a student’s undergraduate education is an internship or co-op. The current practice is for industry to hire students as interns (for example, for a summer) or coops (for example, for a period of six months). You can enhance these experiences by cooperation between industry and academia. Internships and co-ops do not always address students’ specific professional training requirements. However, programs coordinated by academic advisors might address some of the weaknesses identified in the SEA simulation (Criteria 1.1, 1.4, 2.3, and 2.4) and so enhance the students’ practical experience. For selected outstanding students, faculty advisors could plan and facilitate appropriate work experience. A preliminary

Table 1 Simulation evaluation of three academic programs SEA criteria

1.1 1.2

1.3

1.4 1.5

1.6 2.1

Prog-1

No legacy system exposure Rational Unified Process deliverables produced to extent possible within time frame (in a single software engineering course) Internal clients for some teams’ external clients are generally small, local business clients One or two students conduct integration and system tests, with artifacts Informal reviews by client; team presentations and peer evaluations; final products evaluated by clients, faculty, and advisory board No, projects are small- to medium-sized Not all team members deal extensively with client; good experience dealing with several other team members in different roles

2.2

Feasibility analysis in first course

2.3

No exposure to continuous quality improvement Instructor acts as facilitator to team as a whole

2.4

discussion with a company that maintains CMM Level 5 indicated that it’s possible to maintain a dialog between an industry mentor and a student’s advisor (in the academic institution) to plan the broad learning objectives the student should achieve in the internship and co-op programs. The discussion also indicated that most of the aspects listed as measures for the SEA are possible and that the apprentice could achieve many of them over a one-year period.

A

lthough the client-sponsored projects we describe in this article meet many of the SE professional’s development needs, we also note their limitations. However, such projects’ benefits outweigh the costs, both for the student and the client organization. The student benefits from client-sponsored projects by gaining realworld experience. The only cost to the client organization in sponsoring a project is the time the project sponsor spends in communicating requirements, reviewing specifications, and evaluating the final product. (At times, the cost might also include equipment

Prog-2

No legacy system exposure Standard artifacts produced to extent possible within time frame (double-credit capstone course) External clients, mainly small- and medium-sized industries, located by the students Both students conduct testing, with artifacts Often worked at client’s site: formal reviews with client, and final products evaluated by clients, faculty, and advisory board No, projects are small- to medium-sized The two team members interact extensively with client but miss interaction with larger team

Prog-3

No legacy system exposure Standard artifacts produced both in two SE courses and a two-quarter capstone course

Clients in both SE and capstone courses, with only external clients in capstone; these include large government and industrial clients. In capstone, one student performs integration and system tests, with artifacts Requirements and user interfaces reviewed by client; team formally reviews all artifacts; team presentations and peer evaluations; final products demonstrated at trade show to local community Some exposure to scaling up in SE courses Team members take different roles during SE courses, giving all members interaction experience (a variation: two teams work on same project and swap artifacts at each phase end) Feasibility analysis in the system analysis In capstone project, students research alternative and design courses solutions with cost considerations No exposure to continuous No exposure to continuous quality improvement quality improvement More individual feedback due to Instructor acts as facilitator to team as a whole small team size

and software licenses.) The client organization benefits by acquiring a working system or a prototype. A possible mutual benefit occurs when the student gains employment in the organization after graduation. Additionally, the coordinated internship and co-op programs address some of the limitations of using client-sponsored projects in the capstone course. Greater levels of academia-industry partnerships help ease the student’s transition to professional. In the co-ops and internships, the faculty advisor can act as a mentor. When options exist, the advisor guides the student in choosing an option with greater scope for skills enhancement. The cost to industry includes the student’s stipend, training overhead, and any additional time for interaction with the academic advisor. The usual benefits are in having a potential pool of skilled future employees to choose from and reduction in orientation time. Additionally, if the student’s internship or co-op is coordinated with the academic advisor, the future employees should have the advantage of a more complete, broad-based experience. The client-sponsored projects and managed internships together can serve as an alSeptember/October 2002

IEEE SOFTWARE

55

About the Authors Ken Surendran is an associate professor in the Department of Computer Science at

Southeast Missouri State University. His research interests include software engineering, security management education, and object technology. He has a BE in electrical engineering from the University of Madras, India, an M.Tech in electrical engineering from the Indian Institute of Technology, and a PhD in applied analysis from the State University of New York at Stony Brook. He is a senior member of the IEEE and a member of the ACM. Contact him at Computer Science Dept., MS 5950, Southeast Missouri State Univ., One Univ. Plaza, Cape Girardeau, MO 63701; [email protected]. Helen Hays is an associate professor in the Department of Computer Science at Southeast Missouri State University. Her research interests include computer science and software engineering education. She has a BA and MA in mathematics from the University of Missouri, Columbia, and a PhD in higher education (with an emphasis in computer science education) from Southern Illinois University, Carbondale. She is a member of the ACM. Contact her at Computer Science Dept., MS 5950, Southeast Missouri State Univ., One Univ. Plaza, Cape Girardeau, MO 63701; [email protected]. Andrew Macfarlane is a software test manager in QCOM Australia. His research interests include quality assurance and IT education. He has an MA in mathematics from Auckland University and an MSc in mathematics from the University of New South Wales. He is the author of Embracing the System: The Systemisation of Business and Life (1stBooks Library, 2001). He is a member of the New Zealand Mathematical Society and American Mathematical Society. Contact him at QCOM Australia, 52 Douglas St., Milton, Brisbane, Australia 4064; andrew. [email protected].

ternative to an SEA. However, in the long run, undergoing a proper apprenticeship program could become a requirement for SE professional practice. By investing in future employees through apprenticeship programs, industry stands to gain more broadly trained SE practitioners, which should translate into greater productivity and higher-quality products. To realize a full industry-sponsored apprenticeship program, we must consider other administrative considerations in addition to the framework. Any comprehensive SEA program will necessarily include administrative components specific to an organization. These components include environment (capability of personnel to train apprentices, supply and demand for qualified employees), implementation (availability of suitable projects, support personnel for administering apprenticeship programs), and motivation (potential benefits to the organization and to the apprentice). For instance, given the SEA criteria in Figure 3, an organization offering such SEA programs would ideally have a proven track record (for example, CMM Level 3) and also the necessary infrastructure to provide mentors and administrators to support the program. For the SEA framework component, SWEBOK provided the underlying concepts and Bloom’s taxonomy was the chosen instrument for analysis. Similarly, an SEA must identify and apply relevant underlying concepts and suitable analysis instruments pertaining to the administrative components, 56

IEEE SOFTWARE

September/October 2002

to flesh out the program details. These require extensive fieldwork and are beyond this article’s scope, but we intend to address them in due course. In evaluating SE’s maturity level, Gilda Pour and her colleagues observed that existing skill development mechanisms for professionals entering the practice are ad hoc and not identified with the SE profession.11 This reinforces the need for proper SEA programs, but their development will be an evolutionary process. The programs could start with a limited number of outstanding students and committed industries that would be willing to take on such an innovative initiative. Requiring an apprenticeship as a prerequisite for professional practice, as other professions have done, might be a first step toward solidifying the SE profession.

References 1. L.W. Anderson and L.A. Sosniak, eds., Bloom’s Taxonomy: A Forty-Year Retrospective, Nat’l Survey on Student Engagement, Univ. of Chicago Press, Chicago, 1994. 2. T. Hilburn et al., A Software Engineering Body of Knowledge, Version 1.0, tech. report CMU/SEI-99-TR004, Software Eng. Inst., Carnegie Mellon Univ., Pittsburgh, 1999. 3. P. Bourque and R. Dupuis, eds., Guide to the Software Engineering Body of Knowledge, IEEE CS Press, Los Alamitos, Calif., 2001. 4. B. Meyer, “Software Engineering in the Academy,” Computer, vol. 34, no. 5, May 2001, pp. 28–35. 5. B. Boehm and V.R. Basili, “Gaining Intellectual Control of Software Development,” Computer, vol. 33, no. 5, May 2000, pp. 27–33. 6. J. Bach and W.M. McCracken, “SE Education: What Academia Can Do,” IEEE Software, vol. 14, no. 6, Nov./Dec. 1997, pp. 26–29. 7. P.J. Denning and R. Dunham, “The Core of the ThirdWave Professional,” Comm. ACM, vol. 44, no. 11, Nov. 2001, pp. 21–25. 8. W.S. Humphrey, A Discipline for Software Engineering, Addison-Wesley, Boston, 1995. 9. K. Surendran and F.H. Young, “Teaching Software Engineering in a Practical Way,” Proc. 13th Ann. Conf. Nat’l Advisory Committee on Computing Qualifications (NACCQ), Hamilton, New Zealand, 2000, pp. 345–350. 10. R. Cannon, J. Diaz-Herrera, and T.B. Hilburn, “Teaching a Software Project Course Using the Team Software Process,” Proc. 33rd Technical Symp. Computer Science Education (SIGCSE 2002), ACM Press, New York, 2002, pp, 369–375. 11. G. Pour, M.L. Griss, and M. Lutz, “The Push to Make Software Engineering Respectable,” Computer, vol. 33, no. 5, May 2000, pp. 35–43.

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

focus

educating software professionals

Educating Experienced IT Professionals by Addressing Industry’s Needs Dale Callahan and Bob Pedigo, University of Alabama at Birmingham

s information technology continues its rapid progress, both educators and employers have recognized the importance of educating the professionals that design, develop, and deploy information systems. IT professionals—including software engineers—must know how to address the diverse needs of stakeholders, stay abreast of the latest technologies, and leverage them to maximize strategic advantage over the competition. These professionals must also balance high efficiency with

A What should students learn in school, and what should they learn on the job? The University of Alabama at Birmingham developed a graduate program specialty in electrical and computer engineering to address industry needs without compromising academic fundamentals. 0740-7459/02/$17.00 © 2002 IEEE

tight timelines, robust feature sets with low cost, and diverse user needs with architectures’ functional limitations. Who teaches IT professionals what to do in this dynamic, diverse environment? Should universities instill these skills, or is that industry’s responsibility? An innovative graduate specialty in information engineering and management at the University of Alabama at Birmingham (UAB) specifically addresses these issues. Who educates our workforce? The age-old battle between industry and academia over what constitutes education and what should be left to on-the-job training has become more intense and less defined in the IT field. Despite last year’s shrinking economy and subsequent decrease in the number of available jobs, highly skilled IT professionals remain scarce.1,2 With the everpresent gap between available jobs and

skilled workers to fill them, industry is pressuring educators to provide college graduates who can enter the workforce ready to produce. However, providing advanced IT skills and knowledge requires preliminary courses that provide the fundamentals. With more and more college-bound students interested in business or other nontechnical career paths, colleges and universities are finding it harder to attract students to science-based curricula. Moreover, industry pressure and declining enrollment squeeze science-based programs of their nontechnical coursework to avoid extending graduation by a semester or year. Similarly, management information systems programs at many universities offer students the opportunity to avoid the advanced mathematics, algorithm design, and hardware circuitry classes central to traditional computer science, computer engineering, and electrical engineering curricula. But September/October 2002

IEEE SOFTWARE

57

With more and more collegebound students interested in business or other nontechnical career paths, colleges and universities are finding it harder to attract students to sciencebased curricula.

MIS programs often fail to provide graduates that can handle the technical challenges of software engineering in their first IT jobs after college. Industry finds training investments the least likely to add to their bottom line. Tight training budgets become political hot potatoes—managers must balance employees’ training requests with the probable return the company will realize from their investment. Top-performing employees often receive training at the expense of newer, less skilled employees who have yet to prove their value to their companies. Employees attend company-paid training courses to acquire the latest technical skills, only to leave shortly thereafter for better-paying jobs. Hence, corporations prefer to hire candidates that already possess the skills and knowledge to succeed.2 For recent college graduates to compete, they must acquire the skills and knowledge during their school years through their curricula, internships, co-ops, or full-time employment balanced with schoolwork. Academia guided by industry To address these issues, the UAB Department of Electrical and Computer Engineering asked industry executives what they expected from college graduates. The department asked executives to define fundamental business skills that the IT candidate pool is generally lacking and to compile a list of topics that needed to be taught. These questions yielded some interesting feedback. First, the executives revealed that IT professionals are primarily lacking technology management skills, rather than software programming or hardware skills. They noted a major drought of capable technical managers to perform uppermanagement duties, specifically at the chief information officer and chief technology officer levels. In general, technology executives with business backgrounds, hired for their business savvy and managerial skill, lack understanding of the information and computer assets they oversee. Because they do not understand IT’s strategic importance in their industry, they set up the company and themselves for failure. To address this problem, many universities offer technical MBA programs to provide future business leaders with some technical

58

IEEE SOFTWARE

September/October 2002

awareness. However, these programs typically lack the engineering principles necessary to prepare future CIOs for the technical decisions they will face. On the opposite end of the spectrum, managers with robust technical skills typically lack the business savvy their MBA peers possess. These “techies” often have little appreciation for business imperatives such as balancing priorities, managing profits, and minimizing risks. Even worse, technical experts often harbor product and technology biases. To them, every problem can be solved with their favorite technology solutions or vendors. Even experienced technical managers sometimes focus more on IT’s classical engineering aspects than its business aspects. The second major revelation from business executives is that they are not looking to hire software engineers, computer engineers, or even business administrators. Instead, they seek information engineers: people capable of participating in and designing business processes while understanding that information is the most essential element of today’s business. Academia has long educated and trained people to engineer automobiles, televisions, aircraft, and computers, but today’s industry asks academia to teach people to engineer information and the processes that control it. UAB worked these topics into courses, from which a new program has evolved: the Master of Science in electrical engineering, with emphasis on information engineering and management (IEM). Balancing theory and reality Academia and industry have long debated the balance between theory and reality in engineering curricula. Industry’s perception that universities teach outdated methods and use impractical study examples is countered by the universities’ need to teach fundamentals before moving on to advanced, industry-relevant material. Hiring managers demand current skills and practical experience of their entry-level employee pool, but universities find little time to squeeze these skills into curricula packed with background courses in mathematics, science, and humanities. MIS programs address this by introducing students to the latest technologies and teaching them to ad-

dress business problems with IT tools. However, MIS programs don’t teach the engineering fundamentals that students need to design efficient, cost-effective information systems from the ground up. Diagnosing problems is difficult (if not impossible) without understanding how information flows through and is used by a system. College graduates with MIS degrees frequently lack the background to succeed in the information engineering job market, leading many to return to school for graduate degrees. But because job candidates with graduate degrees command higher salaries than their undergraduate peers, employer expectations for practical knowledge and skills are higher still. The IEM program’s coursework balances theory and reality by featuring frequent guest speakers and realistic problem-solving exercises to augment the theory textbooks present. A seminar course provides a forum for students and faculty to share experiences. Guest speakers bring immediate relevance to the discussion by raising problems they are facing in their workplaces. The CIO of a local power company talks about the challenges of the energy business and its effects on IT. A telecommunications executive discusses the implications of the fiberoptic industry’s slowdown and how IT solutions could mitigate bottlenecks in the outdated communications infrastructure. Venture capitalists encourage students to take their innovative IT ideas to market. Faculty encourages students to address their own businesses’ problems as topics for class projects. Students practice their skills on real-world problems rather than hypothetical ones. They can implement at their workplaces the solutions they develop in class, providing value to their companies while honing their own practical skills. Software fundamentals For IT professionals to add value, they must understand the fundamental practices involved in the development, deployment, and retirement of information systems. They must be able to identify life-cycle models, understand their differences, and know when to use the myriad tools in their IT toolbox. They must grasp software development concepts such as quality management, requirements analysis, configuration man-

agement, information integration, and software metrics. Information engineers should be familiar with computer-aided software engineering tools, code libraries, and automated testing tools. These concepts are not confined solely to software developers— sound software engineering practices are essential in any IT setting. The managers who purchase IT consulting services and those who implement the resulting products must understand the methods, tools, and standards used to develop the products they purchase. They must be able to distinguish sound development practices from ad hoc coding, identify good software from bad, negotiate solid contracts with software engineering consultants, and make sound business decisions about IT assets. Teaching software engineering fundamentals has historically fallen to corporate training programs, due to the lack (until recently) of software engineering programs in colleges and universities. Computer science curricula have foundations in science and mathematics rather than engineering, and their emphasis has tended toward computer and algorithm design rather than developing software as a business tool. Although software engineering curricula are becoming more common, long-standing corporate IT fundamentals such as project management, requirements analysis, and the software development lifecycle are new academic topics. But software engineering is part of a larger system that includes computer hardware, communications infrastructure, thirdparty applications, and the information that these combined elements manage. The true value in business today is information— whether data or human-intensive “intellectual capital.” Today’s IT systems provide the strategic value of managing the information that makes a corporation succeed. IT managers must understand the technical fundamentals of software, and be able to manage the way information flows through IT systems to add value to the corporation. UAB carefully chose its IEM program’s fundamentals to ensure relevant coursework and to provide themes that are reinforced across the curriculum (see the “IEM Course Outline” sidebar). The coursework emphasizes recognized standards and standards bodies. Students develop and post

For IT professionals to add value, they must understand the fundamental practices involved in the development, deployment, and retirement of information systems.

September/October 2002

IEEE SOFTWARE

59

Regardless of industry, an MBA is the entry ticket for modern business success.

Web pages that comply with World Wide Web Consortium HTML standards. Students plan, execute, and measure software development projects in accordance with the Capability Maturity Model. Texts and class discussions center around the Software Engineering Body of Knowledge.3 Students complete research papers and presentations describing IEEE software standards and certification programs. Faculty reinforce these themes with readings from software engineering community leaders such as Frederick Brooks,4 Richard H. Thayer,5 Andrew S. Tanenbaum,6 Steve McConnell,7 and Grady Booch.8 IEM program graduates emerge with solid knowledge of software and information engineering fundamentals, able to lead their peers in IT organization planning, execution, and management. MBA versus MS Most professionals who wish to advance into management must decide whether to obtain an MBA. Regardless of industry, an MBA is the entry ticket for modern business success. Holding an MBA doesn’t guarantee admittance to management ranks, but lacking one can inhibit progress. However, an MBA does little to provide the technical skills necessary for managing IT professionals or IT assets. Thus, CIOs without a technical background are at an extreme disadvantage. Gaining staff respect is an essential part of being an effective manager. Without this respect, a manager’s best ideas go unheeded and his decisions are secondguessed. Engineers usually value technical education above business savvy, so they are more likely to tolerate a manager with an MS but little business acumen than one with an MBA but no technical understanding. The IEM program provides a graduate degree that addresses the need for IT industry leaders who possess strong technical knowledge rounded out by practical business and management skills. Rather than teaching technical skills to business students, the IEM program teaches business skills to engineers. Students in the IEM program learn to research the IT business environment, monitor the market, write and present business plans, and defend their work to venture capitalists. These experiences prepare IEM graduates for leadership roles in startups as well as established firms.

60

IEEE SOFTWARE

September/October 2002

By focusing on IT’s technical and business aspects, the IEM program aims to fulfill industry’s need for information engineers who can immediately add strategic value to the IT departments of the businesses they enter. These information engineers can tackle management positions from network support to project management to CIO. Computer science versus information engineering As information engineering moves toward becoming a distinct engineering discipline, the debate over whether computer science curricula can adequately address the needs of engineering professionals is heating up. Arguments abound in favor of separating these two areas into distinct curricula. Some assert that establishing a distinct information engineering curriculum is a fundamental step toward maintaining the field as a legitimate engineering specialty. Others argue that the IT industry is not sufficiently diverse to justify multiple IT education programs. Furthermore, few colleges and universities have sufficient funding for two programs, forcing them to choose which path to pursue, or to attempt to combine the two paths into one program. Although both computer science and software engineering require similar depth of knowledge, they approach that knowledge from distinct points of view. It is this fact that dictates distinct curricula for the two disciplines. Frederick Brooks said, “The scientist builds in order to study; the engineer studies in order to build.”4 As he implies, computer science curricula should attract students who wish to gain advanced understanding of the “hows and whys” of computer operation. Computer science coursework supports algorithm designers, programminglanguage authors, and hardware architects. These people dissect all aspects of a computer’s fundamental functions and determine the best ways to make them tick. Engineering curricula, by contrast, teach students to use technology to create solutions and advance our ability to accomplish our goals. Engineers take the building blocks that computer scientists provide and innovate ways of assembling them to solve problems. The IEM program takes a decidedly engineering approach, focusing on using IT sys-

IEM Course Outline Advanced engineering operations (EE684S)

Business systems design and programming Case analysis Managing in service environments Managing product development Needs assessment Project management Projections and forecasting Quantitative methods Risk assessment Specification preparation Strategic analysis Strategic and operational planning System analysis (cost/benefit and cost/effectiveness) Technology transfer The budget process Engineering management of information resources (EE685S)

Global information technology management Image transfer and management Information flow and use Information process and improvement Management of information Management of quality Managing critical information Managing information-intensive change

Governmental regulations and requirements Legal considerations Measuring information systems’ effectiveness and productivity Reengineering and information systems evolution Introduction to computer networking (EE632S)

Communication protocols: Internet and WWW Computer network architectures Network security Networking and communications Telecommunications (LAN, WAN, Internet, intranet) Video and data conferencing Object-oriented design (EE640S)

Database design Database management (relational, object) Databases and data handling Graphical user interfaces Object-oriented design and analysis Operating systems Programming and programming languages Programming standards and processes (SEI CMM, ISO 9000) Software engineering Software quality and testing Technical entrepreneurship I and II (EE686S and EE687S)

Enterprise information architecture engineering (EE657S)

Data warehousing Developing and managing distributed systems Distributed computing Distributed computing architecture Hardware, software, and communications Information architectures Multimedia engineering Technical architecture design Enterprise perspectives in information engineering (EE688S)

Building a responsive information infrastructure Business process redesign Business systems analysis and design Electronic-commerce technical design Ethics

tems and assets to solve engineering problems. Software design is an important aspect of engineering because only properly designed applications can solve softwarerelated problems. Information engineers learn to evaluate IT tools and choose the right ones for the job. They learn to design IT

Customers, vendors, and employees Economics and finance Hiring and developing people Human resource management—general HR topics Interpersonal and leadership skills Management and motivational techniques Marketing and sales Operations Organization management Organizational culture and behavior Organizational dynamics People and organizational management Personnel management Strategic management The negotiating process Venture capital

systems with information flow in mind from the start to maximize the resulting systems’ business and strategic value. They learn to evaluate the IT market and decide which IT tools and solutions to purchase, based not just on the latest technological trends, but on what will bring the best value to the business. September/October 2002

IEEE SOFTWARE

61

Surveys of the first class to complete the IEM program indicate that the curriculum strikes dead center on what students need to succeed in the job market.

A

s any engineer knows, design is one matter, but the final product is the test of success. The IEM program began with the first group of students in August 2000, and these students graduated in May 2002. Did the program meet its design goals? Feedback from these students and their employers is telling. Surveys of the first class to complete the IEM program indicate that the curriculum strikes dead center on what students need to succeed in the job market. Most students indicated that they learned information and skills they never expected to, and all students indicated that their job and career perspectives had broadened. The students were immediately able to apply their coursework on the job. Many completed class projects that were directly related to their companies’ most pressing problems—problems that were never before addressed properly. One student who worked with a software development company devised new quality metrics and standards for upcoming offshore development plans. A student who worked with IBM developed a framework for creating design specifications; today, IBM’s design groups use that framework.9 Other students have developed new software development models, addressed information integration and data-mining issues, and even started new companies developing software products with industry-leading quality assurance procedures. Although most students’ employers responded positively to the students’ activities in the IEM program, some students received negative attention from company management. Some managers did not like having

their IT operations scrutinized using industry-standard models such as cleanroom software development methodologies. Others did not appreciate suggestions for improvement (from newly educated IEM students), such as using new data integration methods to enhance information collection procedures. However, IEM students successfully handled even these negative experiences, due in part to business and interpersonal skills that the IEM program helped them develop and enhance.

References 1. M. Bolch, “The Coming Crunch,” IEEE Eng. Management Rev., vol. 29, no. 3, third quarter 2001, pp. 23–26. 2. S. Stellin, “The Long, Humbling Quest for a Job in Technology,” New York Times, 17 Mar. 2002. 3. P. Bourque and R. Dupuis, eds., Guide to the Software Engineering Body of Knowledge, IEEE CS Press, Los Alamitos, Calif., 2001; www.swebok.org. 4. F. Brooks, “The Computer Scientist as Toolsmith II,” Comm. ACM, vol. 39, no. 3, Mar. 1996; www.cs.unc. edu/~brooks/Toolsmith-CACM.pdf. 5. R.H. Thayer, ed., Software Engineering Project Management, 2nd ed., IEEE CS Press, Los Alamitos, Calif., 1997. 6. A.S. Tanenbaum, Computer Networks, 3rd ed., Prentice Hall, Upper Saddle River, N.J., 1996. 7. S. McConnell, Rapid Development: Taming Wild Software Schedules, Microsoft Press, Redmond, Wash., 1996. 8. G. Booch, Object-Oriented Analysis and Design, 2nd ed., Addisson-Wesley, Boston, 1994. 9. D. Appleby and D. Callahan, “A Framework for Developing Design Specifications for Complex Heterogeneous Information Systems,” Proc. 34th Southeastern Symp. System Theory (SSST 02), IEEE CS Press, Los Alamitos, Calif., 2002, pp. 11–14.

About the Authors Dale Callahan is an assistant professor and program director of the Master of Science in

electrical engineering with emphasis on information engineering and management at the University of Alabama at Birmingham. He received his PhD in electrical engineering from the University of Alabama. He is a senior member of the IEEE and a licensed professional engineer. Contact him at the Univ. of Alabama at Birmingham, 1150 Tenth Ave. South, Birmingham, AL 35294-4461; [email protected].

Bob Pedigo is an electrical engineer in Birmingham, Alabama. He is a graduate of the

IEM program at UAB. His technical interests include quantum computing, power systems, information engineering, and engineering education. He is a senior member of the IEEE and chairs the Alabama section of the IEEE Computer Society. Contact him at PO Box 361841, Birmingham, AL 35236-1841; [email protected]. For more information on this or ny other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

62

IEEE SOFTWARE

September/October 2002

focus

educating software professionals

Integrating Software Engineering in Introductory Computing Courses Grant A. Cheston and Jean-Paul Tremblay, University of Saskatchewan

rom the first computing curricula recommendations developed, educators have realized that data structures are central to much of computing and that students need a strong background in them early on. It is also important to expose students to numerous computing applications so they can see employment opportunities in the field and realize that computing is more than programming.

F An introductory course for undergraduate computing students aims to integrate data structures and software engineering. The software engineering assignments emphasize teamwork, and students use a 10-step process to develop an objectoriented software system. 64

IEEE SOFTWARE

At the University of Saskatchewan, one of our first-year courses, Principles of Computer Science (CMPT 115), presents a breadth-first view of computing that emphasizes data structures, as did our second-year course, Analysis of Data and Language Structures (CMPT 250). About 15 years ago, CMPT 250 moved to the object-oriented paradigm. Toward the mid 1990s, we added some discussion of software engineering to it, particularly in designing large information systems. CMPT 250 corresponds to the Computing Curricula 2001 report’s recommended third introductory-level course (see the sidebar on p. 67). However, some institutions could use it as their second course, if their first course is particularly strong. Course overview Of the Curricula 2001 report’s 12 units on software engineering, CMPT 250 covers all of three core units: software design, soft-

September/October 2002

ware processes, and software validation. In these three units, we cover more than what’s recommended for introductory courses, especially in the software design unit. We also cover the design-by-contract topic of the formal methods unit. For introductory courses, the report recommends some coverage of the application programming interface software tools units. Basic API use is covered in the prerequisite courses for our course. As for software tools, our students use a programming environment tool. However, software educators must exercise care not to overwhelm students with too many tools too early. Figure 1 contains our course’s topics and associated prerequisite structure. We arranged the topics from top to bottom in order of presentation, with the left side showing the software engineering topics. The abstract data types (ADTs) topic includes preconditions and postconditions 0740-7459/02/$17.00 © 2002 IEEE

and leads to a discussion of the design-bycontract paradigm for software specification and development.1 The analysis and design topic includes a 10-step development process for an information processing system. The steps in this process rely heavily on using the Unified Modeling Language (UML)2 to represent various aspects of the evolving system. (We describe our 10-step process in more detail later.) The last software engineering topic is testing. This discussion includes boundary-value and equivalence-class testing from black-box testing, statement coverage and path coverage from white-box testing, and state-based testing from object-oriented testing. All the software engineering topics should be covered in any course like this one, although a lot of flexibility exists regarding the material covered leading into the project. As for data structures, because our course is at the second-year level, Figure 1’s top three data structure topics are mostly review. If we offered the course in the first year, we would spend more time on these topics because much of their content would be new to students, leaving little time for the bottom three data structure topics. Because many of the students are not yet ready for a course based only on software engineering, about half the course is on data structures. Students need more experience writing actual programs, especially using polymorphism and the more advanced features of object-oriented programming and design by contract. Also, students want to get hands-on experience instead of just doing abstract design. However, they aren’t experienced enough to form teams to actually implement the significant systems they design. Therefore, for the modeling exercises and the project, the students do analysis and design but no coding. This gives them the freedom to incorporate many desirable features without having to implement them, which can be time-consuming. To gain practical programming experience, the students implement and enhance various data structures. We alternate between topics in data structures and software engineering. After we cover a topic in an area, students work on an assignment on that topic while we discuss a topic from the other area in class. In this way, students obtain a mixture of

Review of OO programming Polymorphism

Abstract data types and design by contract

Timing analysis

Review of linear data structures Dictionary data structures Analysis and design process

Binary trees

Project lead-in

Better dictionary implementations • hash table • balanced trees

Testing

Direct and Btree files

Graphs

Figure 1. Course topics and the associated prerequisite structure.

concrete data structure programming and abstract systems design. Additionally, we introduce them to reuse and extending existing code. To facilitate this, we have designed and implemented an extensive library of data structures. This allows exercises that entail adding new features to (or developing a variation of) an existing data structure. In the process, students must read and understand existing code and learn to adapt it by defining new descendants and overriding or redefining inherited methods. Modifying and extending existing software helps maintain legacy and evolutionary systems, so students acquire valuable experience. A simplified object-oriented development approach Much effort in the past decade has gone to developing specific methodologies for OO software development. Indeed, software professionals have published some two dozen or more methodologies, and although some are more popular than others, researchers are still doing a lot in this area. Currently, two thirdgeneration OO development approaches appear to dominate: object-oriented process, environment, and notation and UML. OPEN defines a complete OO development process. September/October 2002

IEEE SOFTWARE

65

To give students more guidance in developing a system, we developed a 10-step process.

Initially, UML was just a set of notations and some modeling rules, but developers have proposed a process for it called Unified. The net result is that industry and academia are fast adopting UML’s set of notations as the standard. Furthermore, extensive computerassisted software engineering tools are rapidly becoming available for UML, so we use UML notation. In the analysis phase of software development, we obtain an object model from a real-world domain that we can use to develop a software system. We initially do the modeling at a high level. The high-level results of the analysis phase are expanded during the design phases to include details of human and internal interfaces. We can then use the design phase results to implement the desired system using an OO programming language. Thus, objects and classes are involved in all phases of development. To give students more guidance in developing a system, we developed a 10-step process. The 10-step process In an introductory course we cannot— nor should we—propose a detailed methodology for OO software development. However, we must give students a fairly specific approach—such as our 10-step process— for designing a large software system. 1. Specify systems requirements In this step, end users, domain experts, systems analysts, and possibly other stakeholders help develop initial requirements for the given problem or application. This step in the analysis phase focuses on what the software system will do—not how it will do it. Although requirements specification is important, we only describe it briefly in our course, and we do not expect students to do this task. For any analysis and design exercise, the requirements are given to the students. 2. Determine the system boundary The next task in software development is separating the planned internal computer system from the external world. Determining this system boundary is fundamentally important and involves unambiguously identifying the communication between the system and the external world. Generally, the relevant external world that we model

66

IEEE SOFTWARE

September/October 2002

consists of users, agencies, devices, and other external computer systems. To determine the communication between the system and the external world, we must identify incoming and outgoing data. A context model and associated diagram represent these dataflows to and from the external entities. Historically, using context diagrams predates object-oriented development. Software developers consider it a good idea to use them to document an application’s scope. Developers should exercise care, however, especially regarding dataflows and message passing. Dataflows are considered inappropriate for object-oriented development. Also, they are not very abstract and might be unsuitable for starting development. The same criticisms apply to context diagrams based on message passing. Furthermore, detailed messages probably contain design information. Notwithstanding these criticisms, we have found that, with proper warnings, students find context diagrams useful. We also identify use cases here because they can facilitate discovering the system boundary. Approaches to finding use cases are either actor- or event-based. UML’s use case diagrams are used to represent actors, events, and use cases. 3. Identify objects and classes An important step in object-oriented software development is examining the problem domain and identifying relevant and good abstractions (that is, ADTs). Any poor abstractions developed in this step will propagate through the entire development process and result in poor-quality software. Identifying good abstractions can be difficult, especially for developers new to object orientation. Unfortunately, no current foolproof approach or algorithm exists for finding good abstractions and their implementations (classes). Students usually find it easiest to identify objects first and from these develop abstractions and classes. A first rough cut at finding objects and classes involves examining the document that specifies the problem’s system requirements. In this document, nouns can sometimes indicate important external-world objects. In the early development phase, we emphasize finding these concrete entity objects. Students can find further objects from

Overview of Computing Curricula Recommendations Computing programs have changed greatly since the 1960s. The ACM’s Curriculum ’68 report1 was the result of the first attempt to guide emerging computer science departments in designing computer science programs. Over the next decade, the recommendations in this report became dated; the ACM updated them in 1978 as the ACM Curriculum ’78 report.2 The first-year introductory courses in this report, CS1 and CS2, were key, and computer science professors are still teaching its contents (except for the programming language used) at many North American institutions. In 1991, the IEEE Computer Society and the ACM released the Computing Curricula ’91 report.3 Unlike its predecessors, this report was more comprehensive. Whereas earlier reports identified a standard syllabus for individual courses, the 1991 report divided the necessary body of knowledge for an undergraduate computing program into nine knowledge units. Individual institutions, depending on their specific needs, could customize their own programs to cover these units. One of the report’s goals was to include software engineering topics in introductory computing courses. After some 10 years, however, this important goal is more an aspiration than a realization. The recent ACM/IEEE Computer Society Computing Curricula 2001: Computer Science report is a major revision of the 1991 report.4 It incorporates the computing developments of the past decade and organizes the body of computer science knowledge hierarchically into three levels: areas, units, and topics. At the top level, the report identifies 14 areas using a two-letter abbreviation, such as AL for algorithms and complexity, PF for programming fundamentals, PL for programming languages, and SE for software engineering. Each area comprises units, which are identified by adding a numeric suffix to the area name; PF3, for example, deals with fundamental data structures, SE1 describes software design, and SE6 deals with software validation. A unit is further subdivided into topics; for example, SE1 includes object-oriented analysis and design, design patterns, and component-level design. For software engineering, the report identifies 12 units; eight are core and should be included in any curriculum. The report also defines detailed course implementations and strategies for complete undergraduate curricula. It describes

domain knowledge and by considering entities that must interact with the system. The context model and use case development can often assist in identifying boundary objects in a given application. Students will obtain other kinds of objects—including conceptual objects, event and state objects, interface objects, container objects, and control objects—in later steps. Once they have found the objects, they must categorize

three levels of courses: introductory, intermediate, and advanced. At the introductory level, the report presents six implementation strategies: imperative first, objects first, function first, breadth first, algorithms first, and hardware first. The main text of this article presents a course that best fits the imperative-first and objects-first strategies. The main difference between these strategies is that the objects-first strategy emphasizes objects very early in the syllabus. In each of the strategies, the report gives two separate introductory-level implementations. The first uses the traditional twocourse sequence, and the second covers the material in three courses. Some computing educators believe that they cannot cover introductory-level topics in only two courses and therefore not in the first year. Other educators use this traditional two-course approach. The Computing Curricula 2001 Task Force predicts that the three-course introductory sequences will become increasingly common.4 Each introductory sequence incorporates some of the core topics from several of the 14 identified knowledge areas. One of these areas is software engineering. For both the imperativefirst and the objects-first strategies, at least one of the introductory courses has significant content in both data structures and software engineering. This article’s main text describes our attempt to integrate the software engineering and data structure topics in one reasonably cohesive course.

References 1. ACM Curriculum Committee on Computer Science, “Curriculum ’68: Recommendations for the Undergraduate Program in Computer Science,” Comm. ACM, vol. 11, no. 3, Mar. 1968, pp. 151–197. 2. ACM Curriculum Committee on Computer Science, “Curriculum ’78: Recommendations for the Undergraduate Program in Computer Science,” Comm. ACM, vol. 22, no. 3, Mar. 1979, pp. 147–166. 3. A.B. Tucker et al., Computing Curricula ’91, Assoc. Computing Machinery and the IEEE Computer Soc., Los Alamitos, Calif., 1991; www. computer.org/education/cc1991. 4. Computing Curricula 2001: Computer Science, Assoc. Computing Machinery and the IEEE Computer Soc., Los Alamitos, Calif., 2001; www. computer.org/education/cc2001/index.htm.

them into classes. This will likely involve extensive pruning, because the initial search for objects and classes probably included synonyms for the same concept. During software development, we want to find, invent, or create three broad categories of classes: domain classes, design classes, and implementation classes. Domain classes belong to the problem space, whereas design and implementation classes belong to September/October 2002

IEEE SOFTWARE

67

Students find that interaction diagrams are quite natural to develop, and through this development, various parts of the system become apparent.

the solution space. Because the boundary is usually blurred between the analysis and design phases, design classes might be the most difficult to create. Also, design and implementation classes result from creating an architecture that specifies how we can achieve the desired functionality. Thus, they often arise later in system development. Equally important to finding good relevant abstractions is recognizing poor or irrelevant ones. Just as we need fruitful approaches to help find good classes, we also should examine danger signals that might lead to classes we should reject. This is especially important for students just beginning system objectification. In our course, we discuss several of these signals. In practice, software engineers prefer using only a few approaches to finding objects and classes, but our goal is to expose object-oriented modeling beginners to several approaches. 4. Identify class interactions and features This step helps establish the various ways objects can interact and collaborate. To identify interactions and features, it is often useful to study in detail the use cases found in Step 2. By analyzing each of these as it progresses through the system, students can discover many interactions and features. We find that interaction diagrams are a good way to portray use case interactions. Also, students find that interaction diagrams are quite natural to develop, and through this development, various parts of the system become apparent—for example, the various containers used to store system entities. Our discussion for this step emphasizes the need for an interface to handle communication with each external entity. We also demonstrate using controller classes to coordinate necessary activities, and we model commands by classes to promote modularity and extensibility. The interactions that students discover in an interaction diagram are modeled using inheritance, aggregation, and association relationships. In the next step, students amalgamate the class diagrams to represent the system’s entire structure. This step also identifies some of the attributes and operations of each class. Sometimes we can discover suitable attributes by finding adjectives in the system requirements document.

68

IEEE SOFTWARE

September/October 2002

Essentially, this process involves discovering suitable abstractions—that is, ADTs. At this stage we ignore any implementation details for these abstractions. 5. Group classes into subsystems To manage complexity and produce quality software, developers organize classes in a system into clusters, or subsystems. Each subsystem comprises selected interacting classes or subsystems that collaborate to deliver a desired functionality. For trivial problems, whose class diagrams students can draw on one page, partitioning a system into subsystems might not be necessary. However, even for small problems, and certainly for medium-size and larger problems, identifying subsystems becomes important in software development. This step’s main goal is to partition system classes into subsystems that are loosely coupled and highly cohesive. In developing larger systems, students might have to group classes into subsystems much earlier—perhaps in the first couple of steps. 6. Determine high-level system architecture Analysis results help produce a system architecture that will demonstrate how to realize the system’s functionality. In this step, students map aggregation and association relationships into client-supplier relationships. Additional classes dealing with software and hardware issues might appear. Also, further inheritance taxonomy development can introduce new classes that we cannot trace back to the problem domain. Furthermore, we introduce the three-tier architecture—which comprises presentation, application, and storage layers—as a desirable one. 7. Find more detailed design classes and perform detailed class design This step produces lower-level classes that support the classes identified in Step 6. Students might obtain these new classes directly from existing class and application libraries. Another possibility is to adapt existing library classes to the current problem. This step usually involves bottom-up design, which emphasizes class design, including each class’s full interface or services specification. Students update UML class diagrams for the design with the additional details. An important part of this step is identify-

ing the best data structure for each container. We do this by matching the required application operations with the possible data structures’ operations. Thus, students must be very familiar with the operations of the various data structures and their efficiencies. We cover this information in the data structures part of the course, which covers arrays, linked lists, stacks, queues, trees, and various dictionaries. The dictionary ADT is probably the most important because it occurs in most information processing systems. Thus, knowledge from the data structures part of the course is important for this part of software development. 8. Write code for first working prototype In this step, the interaction (sequence or collaboration) diagrams can again prove useful. The interaction diagram for a use case shows a sequence of interactions: one object’s sequence of interactions with other objects becomes the operations sequence for a method of the object, and each operation in the sequence is an attribute access in—or a method call on—another object. Thus, by tracing through all the interactions, students can identify further attributes and methods and develop the methods’ code. 9. Review system for quality considerations This step helps improve the architecture from reuse and quality viewpoints. Although reuse is always considered when students perform the previous steps, the system is now polished with respect to its possible reuse and quality. This activity will often result in extending and revising existing inheritance taxonomies. Students might change subsystem organization to improve coupling and cohesion. 10. Refine coding for first working prototype and perform testing As its name implies, this step involves code refinement and system testing. The testing requires designing test cases for each class, each subsystem, and the entire system as a whole. Course emphasis In our course, we emphasize some steps more than others, particularly the analysis in Steps 2 through 5 and the design in Steps 6 and 7. The students don’t actually do Step

8—implementing a significant system. We discuss it in class, and of course students implement data structures. We cover Step 10’s testing component separately, rather than covering the complex task of testing a complete system. Finally, in class we discuss using several design patterns to improve system quality. We also illustrate how you would use the patterns in a case study. The patterns we discuss include Expert, Creator, Controller, Polymorphism, Law of Demeter, Iterator, Handle, Whole-Part, and Facade. Our 10-step approach is for developing a system; it does not include the important operation and maintenance phase. Also, we must inform students that they wouldn’t usually perform the development steps sequentially. Indeed, not only would they perform each step incrementally with repeated refining, but they can do several steps concurrently. In software development, we do some parts from the top down, generate others from the bottom up, and use a mixed (top-down and bottom-up) strategy for others. Particularly, we would often do Steps 8 through 10 iteratively while developing successive versions of a system prototype.

Software engineers must be able to model an application and use that model to design and implement a system.

Assignments and projects Software engineers must be good programmers, well versed in data structures and algorithms, and fluent in one or more programming languages. Also, they must be able to model an application and use that model to design and implement a system. Moreover, software engineers will be involved in programming in the large, where many developers collaborate to develop a large system. On such projects, the software engineer is part of a team and therefore needs communication and interpersonal skills. He or she must schedule work for themselves and others. Students best learn analysis and design by doing it, so we give them challenging and worthwhile exercises to learn these skills. Normally, the first one is a fairly simple information processing problem with a few obvious use cases. Students are only required to analyze and design a system without doing an implementation because the system is usually too large and complex to implement. As we described earlier, they do the modeling using a UML subset. The design should include the most appropriate September/October 2002

IEEE SOFTWARE

69

Students will hopefully realize that software engineering is not just coding.

70

IEEE SOFTWARE

data structure for each container in the solution. Thus, the data structures section integrates well with the design section. Through these exercises, students will hopefully realize that software engineering is not just coding. They should learn that with a good, detailed system design, the coding is fairly straightforward—although sometimes long and laborious. Also, they start to see the difference between a systems analyst and a programmer. Ideally, students should already be familiar with the application in the exercise. Some applications we have used are a tennis court booking system, a rental company, and the sale of used textbooks by consignment. Students should do the initial analysis and design exercise individually to ensure that they become familiar with the steps and associated artifacts produced. The grading of such an exercise emphasizes a design’s various components, including the use case list, the context diagram, a few interaction diagrams, the corresponding class diagrams, a composite class diagram, and the inheritance hierarchies. Only a few marks go toward design quality, because students are just learning to design. Once an application is discussed in class, in the text, or in an exercise, we can assign exercises that add further operations or commands. This is particularly easy if objects model the commands. Often, only the design is done for an additional command, but it helps if a working version of the system is available. We developed two fully operational systems, which let students study a final system and actually add new commands to it. It is also worthwhile to have a final project where students work in teams to analyze and design a larger system. In a larger system, many of the use cases are not so obvious, and there is usually more than one workable design. Through teamwork, different members often develop somewhat different designs. As a result, the team must work together to analyze them and determine the best one to develop in more detail. This enhances the students’ ability to work at a more abstract design level. To ensure that every student participates in the work, teams are usually restricted to three (sometimes two) members. Large teams are unnecessary because the students do no implementation. Projects we’ve used include

September/October 2002

a university library, a university bookstore, and a student registration system. Although grading an undergraduate student project is not a trivial task, a graduate student in software engineering should be able to do it. In doing the grading, the student will see a variety of obtainable designs, including the poor ones, so that he or she can learn to avoid them. Student response Initially, students are somewhat overwhelmed by analysis and design problems. However, they can begin by identifying use cases and developing an interaction diagram for each one. Provided that the first problem is not too difficult, they can generally obtain a reasonable design. Often, some of the use cases are discussed in the tutorials associated with the course, and students handle the other ones. After completing one problem or application’s analysis and design, students gain confidence and can tackle much larger applications. In course evaluations, students usually indicate that analysis and design is their favorite part of the course. Often, students are proud of their project designs and sometimes reference them in their resumes. Support When we first decided to include more software engineering in our course, we realized that there was no suitable text. Most first- and second-year data structure texts discuss ADTs, but often such discussions are superficial. Similarly, most introductory texts discuss testing, but the discussions often omit the details of specific test-case generation approaches. With respect to software design, we could not find a text at the firstand second-year levels that described an object-oriented software development process. There are several texts for a third-year level course on object-oriented software development, but they assume too much background knowledge to be useable by firstand second-year students. Thus, we wrote our own text, of which there are now two editions. One edition uses Java to show the ideas’ implementation;3 the other uses Eiffel, a language that is not as well known.4 Eiffel is a strongly typed, object-oriented language with a Pascal-like syntax. It supports abstract classes, multiple inheritance, generic

About the Authors Grant A. Cheston is a professor in the computer science department at the University

types, and design by contract with assertions popularized by Bertrand Meyer.1 To assist students in our course, a senior student runs a weekly tutorial session to answer questions about the current assignment and do additional examples similar to the required ones. When students are doing their initial analysis and design assignment, the senior student usually does part of the application in tutorials to demonstrate what is involved while the students complete the rest. This way, they see an example that is closely related to their assignment.

T

hrough our course, students obtain most of the fundamental concepts of data structures, modeling, software design, and software validation. Students also learn the finer points of object-oriented programming, such as multiple inheritance and programming with assertions. This is important in that it ■ ■

■

■

■

Prepares the student for the real world Shows that software engineers not only do programming in industry but certainly are expected to do analysis, design, and testing Demonstrates that using patterns in software development can produce more loosely coupled, highly cohesive systems that will be more reusable, modifiable, and extendible Emphasizes the fact that a software engineer is often part of a team and therefore needs communicational and interpersonal skills, which can greatly affect team productivity Emphasizes that software engineers must be able to design and implement small collections of classes as well as work with a team on a large, complex system

In the future, we plan to have students use a modeling tool to draw the UML diagrams. We would also like to more fully investigate the problems that arise when introductory students work in teams. Furthermore, we would like to measure the extent to which the analysis and design aspects of this course prepare students for subsequent courses and employment.

of Saskatchewan. With Jean-Paul Tremblay, he has coauthored the Eiffel and Java editions of Data Structures and Software Development in an Object-Oriented Domain (Prentice Hall, 2001 and 2003). His research interests are in the graph algorithms area. He has a PhD in computer science from the University of Toronto. Contact him at the Dept. of Computer Science, 57 Campus Dr., Saskatoon, Saskatchewan, S7N 5A9, Canada; [email protected].

Jean-Paul Tremblay is a computer science professor at the University of Saskatch-

ewan. In addition to the books coauthored with Grant Cheston, he coauthored Discrete Mathematics: A Computer Science Perspective (Prentice Hall, 1996) and several books in the McGraw-Hill Computer Science Series. He has a PhD in computer science from the Case Institute of Technology. Contact him at the Dept. of Computer Science, 57 Campus Dr., Saskatoon, Saskatchewan, S7N 5A9, Canada; [email protected].

References 1. B. Meyer, Object-Oriented Software Construction, 2nd ed., Prentice Hall, Upper Saddle River, N.J., 1997. 2. M. Fowler and K. Scott, UML Distilled: Applying the Standard Object Modeling Language, Addison-Wesley, Boston, 1997. 3. J.P. Tremblay and G.A. Cheston, Data Structures and Software Development in an Object-Oriented Domain: Java Edition, Prentice Hall, Upper Saddle River, N.J., 2003. 4. J.P. Tremblay and G.A. Cheston, Data Structures and Software Development in an Object-Oriented Domain: Eiffel Edition, Prentice Hall, Upper Saddle River, N.J., 2001.

For more information on this or ny other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

Good news for your in-box.

Sign Up Today for the IEEE Computer Society’s e-News Be alerted to • articles and special issues • conference news • submission and registration deadlines • interactive forums

Available for FREE to members.

computer.org/e-News September/October 2002

IEEE SOFTWARE

71

focus

educating software professionals

Teaching Teamwork Thomas B. Hilburn, Embry-Riddle Aeronautical University Watts S. Humphrey, Software Engineering Institute, Carnegie Mellon University

he software industry needs engineers who know how to produce quality products on schedule. Because computer science programs do not typically teach engineering concepts or practices, students often start their professional careers with little understanding or appreciation of the discipline needed to build quality products or the methods needed to keep projects within cost and schedule constraints. As a result, industrial software groups generally work without plans and have serious schedule and quality problems.

T Software developers need training in software engineering, but few computer science programs offer such courses. Project courses that use a defined team process prepare students to build quality software products under real-world constraints. 72

IEEE SOFTWARE

To meet industry’s needs, many computer science programs now offer software project courses.1 Course methods vary, however, and there is little agreement as to what the courses should cover. Most academic programs have at least one software engineering overview course and a few offer several courses on the subject. At Embry-Riddle Aeronautical University, students learn the Personal Software Process2,3 in their first year, and the Introductory Team Software Process4 in their second and third years. The PSP uses a phased development paradigm to teach students fundamental engineering practices. The TSPi course teaches teamworking methods and exposes students to the practical issues and problems of teambased product development. Many other schools are taking an approach similar to ERAU’s, and some schools have even instituted software engineering degree programs.5 This article reports on experiences with the TSPi course at ERAU and at several other institutions.

September/October 2002

Software project courses In the latest draft of volume II of Computing Curriculum 2001 (CC2001),6 the CC2001 Task Force states that to help students acquire professional skills as undergraduates, all computer science programs should include ■ ■

Early opportunities for teamwork A complex project (usually undertaken in the senior year) designed and implemented by small student teams

Software project courses range from firstyear two- or three-student teams working on brief (few-week) projects to one-year seniorlevel courses in which student teams work for real or quasi-real customers. At some universities (Carnegie Mellon, Georgia Tech, and the Milwaukee School of Engineering, for example), students join ongoing laboratory- or studio-based software projects that might involve both development and maintenance.7–9 One hotly debated issue is the degree to which project courses should incorporate 0740-7459/02/$17.00 © 2002 IEEE

Product need

Cycle 1

real-world experiences. One side advocates exposing students to real projects with vague and unstable requirements and unrealistic (or unmanageable) schedule constraints—a “sink or swim” strategy. The other side promotes developing toy projects with stable requirements and realistic schedules in a familiar development environment. Most project courses fit somewhere between these two extremes. Designing a project course The academic environment places some special constraints on project courses. First, you must restrict the course to a single term or plan, manage, and coordinate it over several terms. The dynamics of typical student populations make it difficult to maintain stable team membership for more than a single term. Moreover, academic environments can rarely find users to test multiple versions of the completed products. The second problem concerns team formation. Computing faculty rarely have training or experience in creating effective software teams. Forming and building teams involves ■ ■ ■ ■ ■

Selecting team members Assigning team roles Building cohesive units Assessing progress Providing meaningful advice and guidance

Third is the problem of project oversight and guidance. A software project course should not focus solely on producing working computer programs. It must teach the proper use of accepted engineering practices, such as project estimation and planning, requirements analysis and specification, and high-level and detailed design. These courses must also address practical project issues— for example, task scheduling and tracking, quality measurement and management, and phased development and control. Unfortunately, the typical undergraduate computer science curriculum does not include these practices, and few faculty can provide practical and specific guidance in these areas. Without explicit guidance, most student projects become lessons in how not to develop software. Indeed, most of today’s software developers learned to program by learning a programming language, with little or no guidance on disciplined methods or quality practices.

Cycle 2

The Introductory Cycle 3 Team Software Launch Launch Process Launch Strategy To address these inStrategy dustrial and academic Strategy Plan Plan problems, the Software Plan Requirements Engineering Institute has Requirements Requirements Design developed a family of Design process improvement Design Implementation methods for individuals, Implementation Implementation Test teams, and organizaTest tions. The PSP helps stuTest Postmortem Postmortem dents and professional Postmortem software engineers organize and plan their work, track their proFinished product gress, manage software quality, and analyze and improve their performFigure 1. Introductory ance.2,3 It provides the necessary foundation Team Software for subsequent team-working courses. More Process structure and than 30 institutions now offer introductory flow. Development and graduate-level PSP courses. teams produce the Once trained in the PSP, students take a software product in software project course using the TSPi, which a series of cycles, is an academic version of the Team Software integrating and Process that industrial software teams use. testing the completed TSPi support materials include a textbook, an system in the final instructor’s guide, a support tool, and all the cycle. scripts, forms, standards, and methods needed to develop quality software products.4 The TSPi process divides a software development project into development cycles, with the team producing part of the product in each cycle. Figure 1 shows the TSPi processes and their cyclic structure. Depending on the course constraints, students can complete two or three cycles in a onesemester course. In the final cycle, students integrate and test the finished system. Process description The TSPi starts with team building. During project launch, students form teams (of four to six students), establish team structure, and produce a project plan. This provides the essential foundation for a successful project. Teams then set measurable goals and objectives. For example, a team might state a quality goal as ■ ■

Team goal—Produce a quality product. Measure 1—More than 80 percent of the defects will be found before the first compile. September/October 2002

IEEE SOFTWARE

73

Table 1 Introductory Team Software Process roles and responsibilities Role Team leader Development manager Planning manager Quality/process manager

Description Leads the team and ensures that engineers report their process data and complete their work as planned Leads and guides the team in product design, development, and testing Supports and guides the team in planning and tracking its work Supports the team in defining the process needs and establishing and managing the quality plan

Support manager

Supports the team in determining, obtaining, and managing the tools needed to meet its technology and administrative support needs

■ ■

Measure 2—No defects will be found during system test. Measure 3—At project completion, all product requirements will be correctly implemented.

One of the most important factors in effective team building is defining clear roles for each team member. Table 1 describes the TSPi roles. The process details the responsibilities and activities for each role during each phase. Thus, at the beginning of the project, students understand their roles and know what is expected each week. The TSPi process covers all of the essential elements for effective quality management— for example, test planning during the requirements and design phases and structured test plan reviews. Each team formally inspects the requirements and design specifications and holds both personal code reviews and peer code inspections of each product unit. Students receive review and inspection process documentation that includes process scripts, guidelines for developing review checklists, and forms for recording and reporting inspection data. Because all TSPi artifacts receive quality reviews and inspections, student teams generally produce quality products. During the planning cycle, teams produce a comprehensive plan that includes ■ ■ ■ ■

■

■

74

IEEE SOFTWARE

A list of the products to be produced and their estimated sizes A list of tasks to be completed and the team member responsible for each task Estimated effort to complete each task A week-by-week schedule that identifies the tasks to be completed and the available student work hours A quality plan that estimates defects to be injected and removed in each development phase A template summarizing the product’s estimated and actual size, effort, and defect data

September/October 2002

TSPi teams develop and track their plans with a spreadsheet tool developed by the SEI. This tool supplies a host of metrics with which faculty and students can track and assess process fidelity, product quality, plan accuracy, and project status. Faculty and student feedback More than a dozen university computing programs have used TSPi in software project courses. Several schools, including Auburn University, Carnegie Mellon University, and the University of South Carolina, have shared with us their experiences using the TSPi in both undergraduate and graduate courses. In most cases, instructors presented TSPi material in a laboratory setting using a combination of lecture and informal coaching. While course results varied considerably, all faculty felt that the TSPi course was worthwhile. Many also found that they could adapt the material to the level, background, and sophistication of their students. Not surprisingly, instructors who had used the TSPi several times reported greater success and satisfaction than those who taught it only once. Early problems resulted from defects in the initial version of the TSPi tool as well as its size—it was too big to easily distribute for team use. Also important was students’ PSP preparation. The better their preparation, the more likely they were to appreciate and properly use the TSPi process. The following quotes are faculty reactions to the TSPi process and course. ■

■

“TSPi gives very good material and insights about how to help students organize as a team: role definition, visible commitments, meeting organization, schedule, and so on.” “I think the TSPi is a very good vehicle for teaching a project course once the students have PSP knowledge… it provides a lot of good, usable structure that students can pick up, try out, and take away from the course to their jobs.”

Table 2 TSPi data from 18 teams ■

“I will never teach a team project course without TSP again!”

Metric Size (LOC), cycle 1 Size (LOC), cycle 2 Effort (hours), cycle 1 Effort (hours), cycle 2 Productivity (LOC/hr), cycle 1 Productivity (LOC/hr), cycle 2 Defect density (def/KLOC), cycle 1 Defect density (def/KLOC), cycle 2 Test defect density (def/KLOC), cycle 1 Test defect density (def/KLOC), cycle 2 Effort estimation error, cycle 1 Effort estimation error, cycle 2

In general, students were most positive when they studied PSP in their first year and continued to use it in subsequent courses. When instructors introduced PSP and TSPi late in the curriculum, more students objected to the required discipline. Also, students who are already accomplished programmers and have never experienced the industrial quality and management problems the PSP and TSP address, can be somewhat negative, especially at the beginning of the TSPi experience. The most common complaints about PSP and TSP courses concern data gathering, planning, tracking, and quality management. Although these disciplines are not easy to learn or practice, the best industrial work requires them. Students who do not learn these planning and quality management skills through their coursework will rarely learn them on the job.

Data analysis The computer science program at ERAU introduces the TSPi in a second- or third-year undergraduate team project course focusing on software engineering fundamentals. Prerequisites are programming experience in an object-oriented language (such as Ada, C++, and Java) and PSP experience. The TSPi course aims to expose students to software team project issues, principles, methods, and technology. Because about one-third of the course time is devoted to software engineering lectures and discussions, there is only time for a two-cycle process. Each team follows the TSPi process but adds a customer review at the end of the first cycle and a more formal customer acceptance test at the end of the second, with the instructor acting as the customer. For consistency, we took the TSPi data in the following analyses from the ERAU courses. Since Fall 1998, ERAU has offered the TSPi course each semester. Through Spring 2001, 42 teams had completed projects using the process. Each team collected data on task completion time, defects found in review, compilation, testing, and size of resulting artifacts. They entered these data into

Minimum 485 269 99 72 2.2 2.0 56.5 44.2 2.5 0.0 0.3% 2.7%

Average 692 610 211 119 3.5 5.8 105.1 93.0 9.6 25.4 10.8% 30.9%

Maximum 1,015 1,112 251 217 5.0 12.3 174.6 219.1 20.0 83.0 31.8% 53.0%

the TSPi tool to produce metrics for tracking progress, planning support, and assessing product quality and team performance. In analyzing data from the 42 ERAU teams, we found several errors and omissions. Also, data from the course’s first year were organized differently from other years or were not available. For this article, we identified 18 teams that reasonably represent the work you can expect when students follow the TSPi process properly. Table 2 summarizes the data from these 18 teams. While they produced slightly more code in cycle 1 than in cycle 2, the cycle-2 code was generally more complex. We attribute the greater productivity in cycle 2 to the teams’ better understanding of the TSPi and increased efficiency. Defect density did not significantly improve from cycle 1 to cycle 2, but test defects increased. This is partly due to increased code complexity but also likely indicates that teams discovered defects in the second cycle that they had missed in the first. Most importantly, the teams had not yet developed effective precompile reviews, as evidenced by cycle 2’s average code-review rate of 309 lines per hour—well above the recommended maximum rate of 200 lines per hour. This area requires improvement. Teams estimated effort quite accurately for cycle 1 but overestimated effort for cycle 2. This occurred because the teams based their cycle-2 plans on cycle-1 data, but their average productivity improved 66 percent from cycle 1 to cycle 2. In a three-cycle course, the students would have recognized and corrected the problem. Figure 2 shows the effort distribution over the development phases. The 15.6 percent of effort devoted to requirements is appropriate for the problem size September/October 2002

IEEE SOFTWARE

75

Figure 2. Effort by phase (in percentages). “Miscellaneous and analysis” includes the launch and postmortem phases as well as the team’s preparation of a final report and presentation. “Plan” includes both the strategy and planning phases.

Test 13.1 Implementation 13.1

Miscellaneous and analysis 31.5

Design 13.0

Plan 13.8

Requirements 15.6

and complexity, but the 13 percent spent on design is much too low and is probably one reason for the high test defect density. The course introduces a simple object-oriented design methodology, but students do not get intense design experience until a follow-up junior-level software analysis and design course. Figure 3 shows the average defects injected and removed in each project phase. As shown, teams found and removed about 80 percent of the defects before testing began. This is remarkable for sophomore and junior student teams.

Student assessment After each cycle, students completed an anonymous survey. Students, especially those who had been involved in previous team projects, were generally positive about their experiences. About 75 percent of the students were positive about TSPi, and more than 90 percent felt that working on a team project was a worthwhile learning experience. The survey also included questions about what students liked most and least about the course. Overwhelmingly, students were most positive about having clearly defined roles and a process that describes the tasks for each project phase. The biggest complaint was the Figure 3. Defects by amount of paperwork involved. Although phase. Students most students grudgingly admitted that docdiscovered and umentation, data collection, and data analyremoved most defects sis were important, they did not enjoy it.

Course suggestions Designing and implementing a software project course can be challenging and demanding. Faculty who have taught such courses will tell you that a successful teacher must work hard, properly plan and prepare, track team progress, and assess both the team members’ work and their products. To improve the likelihood of a successful effort, you should follow several guidelines. First, clearly identify course goals. Incomplete or fuzzy goals lead to incomplete or fuzzy results. Second, if the course is time-restricted or will represent students’ first team project experience, use a modest and well-defined problem. Real-world, complex problems can doom a team to frustration and failure. Third, use a defined team process for the project work. We believe the TSPi includes all the elements needed to build, guide, and support effective teams. However, whatever process you use, be sure it includes ■

■ ■

■

A detailed written description of the project and the process to be used. This should include support for planning, tracking, and configuration management; procedures and standards for requirements, design, implementation, and testing; and methods and procedures for inspections and reviews. A description of team roles and associated responsibilities. A list of data teams should collect for use in tracking and assessing performance and product quality. A cyclic development approach. We believe incremental development is not only good engineering practice but is a pedagogically sound method of teaching and learning.

Defects (in percentages)

before testing began. 90 80 70 60 50 40 30 20 10 0

Defects injected Defects removed

Re

76

ir qu

em

en

IEEE SOFTWARE

ts

De

sig

n Im

p

e lem

nta

September/October 2002

tio

n

Te

st

Fourth, enforce process discipline (phased development, data collection, documentation standards, reviews and inspections, and so on). After you have decided what activities and deliverables are important, make sure your students follow the process. Students do not easily accept or understand the need for process discipline. Most students (and many software engineers), for example, do not enjoy documenting project work, but few will debate the need for it. As teachers, we must motivate and help our students understand the need for discipline. Easily acquiescing to

Summer Faculty Workshop on the PSP and TSPi For the past six years, the Software Engineering Institute has sponsored a summer faculty workshop on the Personal Software Process (PSP) and the Introductory Team Software Process (TSPi). In July 2002, the SEI and the US National Science Foundation cosponsored a one-week workshop, “Teaching a Software Team Project Course: Improve Team Performance by Teaching Engineering to Teams,” at Southern Polytechnic State University in Atlanta, Georgia. The workshop aimed to prepare faculty participants to

complaints and criticism can cause teams to lose confidence in the process and produce a chaotic “process du jour” work environment. ■ Develop and implement a software project course or activity that teaches students to Finally, the course instructor work effectively in teams to deliver a quality product on time and within budget should move from the lecturer ■ Develop courses and curricula that include software process concepts and practices mode to a coaching mode. Coaches not only explain methods and procedures for carrying The workshop also gave participants first-hand experience with both PSP and TSPi. You can find details about future workshops at www.sei.cmu.edu/tsp/workshop.html. out tasks, they also track and observe team and individual performance and help team members improve. Moreover, they do not throw teams into the big game (an industrial-strength project) before Several reviewers kindly assisted us in preparing they have learned and practiced the basics this article. We very much appreciate the helpful com- About the Authors ments of Julia Mullaney, Bob Musson, and Bill Peter(with well-defined, modest projects). son. We also thank the IEEE Computer Society reWatts Humphrey’s biography Before initiating a TSPi-based course, viewers for their insightful comments and suggestions. appears on page 24. you should confer with other faculty who Our thanks also go to the IEEE Computer Society editorial staff for their professional assistance in produc- Thomas B. Hilburn’s biography have taught the process. We recommend ing the finished article. appears on page 24. that you attend the TSPi faculty workshop The Personal Software Process, PSP, Team Softoffered each summer (see the “Summer Fac- ware Process, and TSP are service marks of Carnegie ulty Workshop on the PSP and TSPi” side- Mellon University. bar). We have also found that the PSP and TSPi are most effective when integrated into an entire software curriculum rather than treated as isolated technologies. References

A

t Embry-Riddle, we have had excellent success using TSPi in introductory software engineering and senior design courses. Of the 42 TSPi teams in the introductory course, those that carefully followed the process had the best performance and produced the best products. However, all teams finished on time with working products. This is in marked contrast to our previous experiences with team projects where many were late or did not produce a working product. Thus, “teaching teamwork” can make a significant difference.

Acknowledgments We would like to thank the following faculty who shared with us their TSPi experiences: Bob Cannon (University of South Carolina), Rubby Casallas (Rochester Institute of Technology), Jorge Diaz-Herrera (Southern Polytechnic University), Greg Hislop (Drexel University), Keith Olson (Montana Tech), Mark Sebern (Milwaukee School of Engineering), Jim Tomayko (Carnegie Mellon University), and David Umphress (Auburn University).

1. S. Jarzabek, ed., “Teaching Software Project Courses,” Forum for Advancing Software Eng. Education (FASE), vol. 11, no. 6, June 2001, www.cs.ttu.edu/fase. 2. W.S. Humphrey, A Discipline for Software Eng., Addison-Wesley, Boston, 1995. 3. W.S. Humphrey, Introduction to the Personal Software Process, Addison-Wesley, Boston, 1997. 4. W.S. Humphrey, Introduction to the Team Software Process, Addison-Wesley, Boston, 2000. 5. K. Modesitt, “Annual Survey of SE Academic Programs,” Forum for Advancing Software Eng. Education (FASE), vol. 10, no. 11, Nov. 2000, www.cs.ttu.edu/fase. 6. Joint IEEE Computer Soc./ACM Task Force on Computing Curriculum, Computing Curriculum 2001, vol. II, Dec. 2001, www.computer.org/education/cc2001. 7. D. Garlan, D. Gluch, and J.E. Tomayko, “Agents of Change: Educating Software Engineering Leaders,” Computer, vol. 30, no. 11, Nov. 1997, pp. 59–65. 8. M. Moore and C. Potts, “Learning by Doing: Goals and Experiences of Two Software Engineering Project Courses,” Proc. 7th Software Eng. Inst. Conf. Software Eng. Education, Springer-Verlag, New York, Jan. 1994, pp. 151–164. 9. M.J. Sebern, “The Software Development Laboratory: Incorporating Industrial Practice in an Academic Environment,” Proc. 15th Conf. Software Eng. Education and Training, IEEE CS Press, Los Alamitos, Calif., 2002, pp. 118–127.

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

September/October 2002

IEEE SOFTWARE

77

focus

educating software professionals

Software Process in the Classroom: The Capstone Project Experience David A. Umphress, T. Dean Hendrix, and James H. Cross, Auburn University

magine the traditional college project course: students bunch together in groups, are assigned a sizable software development task, and are told to “have at it.” Instructors expect them to integrate the technical skills they’ve learned in previous courses, learn to work synergistically as a team, plan and track their work, satisfy their customer—and produce sound software. Yet, more often than not, projects so framed teach their participants yet another way not to develop software.

I A process-oriented perspective on large student projects guides students in integrating end-toend life-cycle skills and provides consistency of experience among projects. The authors discuss what they learned after conducting 49 capstone projects. 78

IEEE SOFTWARE

Some in the academic community might argue that such reality-based software development reflects industry practice. This might be an accurate generalization of the past, but recent trends suggest that a purposeful and disciplined approach to software development is more effective than an extemporaneous one.1 Organizations (such as Raytheon2 and General Dynamics3) are making conscious efforts to transform projects from initiation rites that yield nexttime-I’ll-do-it-differently lessons into being ventures that promote this-worked-and-I’lldo-it-again experiences. They see software development as more than a grab bag of technical undertakings—they envision it as the purposeful orchestration of technical and nontechnical activities. We can mirror this vision—and hopefully the successes—in college project courses if we are prepared to accept what industry is learning. The lesson

September/October 2002

is that software development requires discipline, and we can foster discipline using well-conceived software processes. We outline our rationale for moving from a product-oriented approach to a processoriented one, our experiences in determining suitable process weight, and lessons we learned in attempting to make reality-based software experiences less painful and more real. Using processes in the software engineering classroom Using processes to develop software in the college classroom has mirrored industry history (see the “A Brief Process History” sidebar on page 81), although on a much more attenuated and compressed time scale. While most projects adhere to the classical school of development, there is a growing recognition by instructors of the usefulness 0740-7459/02/$17.00 © 2002 IEEE

of process, however meager, for student projects in the academic world. Although much can be said for the feelings of freedom and independence gained by letting students build software as they please, the simple fact is that most students have not been taught beginning-to-end project skills. Current computer science and software engineering curricular models ensure students have taken a variety of classes relating to programming. However, this does not assure that they have integrated the knowledge they gained in coursework into a comprehensive and functioning whole, nor does it guarantee that they have the social skills required for a successful project. Using software processes in the classroom helps in three ways. First, the processes describe the tasks that students must accomplish to build software. Suitably detailed, a process model can describe the life-cycle activities: their sequence and starting and stopping conditions. It can also define the project’s artifacts, outlining their appearance, content, and the method students should use to build them. In a sense, processes let students focus on the creative task of building software by defining activities instead of wasting time inventing them. Second, processes can give the instructor visibility into the project. Viewed this way, they describe the rules under which the project operates. The instructor can query students on their progress down to the granularity of the process description; students follow the rules outlined in the process to report status to the instructor. Third, processes can provide continuity and corporate memory across academic terms. Instructors can build solutions to general project problems into processes, thus improving the project’s educational worth over time. Capstone projects We have concluded 49 capstone software development projects, 27 of which used various process-oriented approaches. Each project entailed a team of three to five students developing new software or making significant enhancements to existing software. All the projects involved a customer who used the product after delivery. We conducted the projects at the graduate and undergraduate levels. Graduate teams worked for a year, while undergraduate teams worked for a sin-

gle academic term. On average, graduate students had five years of software development experience; undergraduates generally had no professional exposure. All students had taken programming-intensive classes and at least one design class before starting their project. Most students had completed a software process class. Projects ranged widely in domain, programming language, and hardware platform; examples included constructing software tools in Java for desktop computers, developing controllers in C for embedded hardware, and building project-tracking tools in Visual Basic for handheld computers. Software process trends in student teams Our outlook on process follows the history of industry trends; that is, we started with no process, injected too much process, and then backed off to what we felt was a satisfactory balance of discipline and chaos. Indeed, we found that injecting process into the academic environment requires the same technology transition skills and tools used in industry. Our traditional capstone projects followed the classical school of software development. Although each team developed software for widely differing customers and domains, the projects were similar in nature: They captured requirements and designs, but seldom validated or verified them. Teams ignored designs in the heat of final deadlines. The bulk of the project work came at the end of the project, and was normally done by a small subset of the team working exhaustive, long hours. Project documentation was pretty, but generally vacuous. It was often relegated to the weaker members of the team as a way of providing them a task that got them out of the way of the software construction. Requirements were scaled down from the initial promises to the customer, usually near the project’s end. The product delivered by the team could not fully pass customerdesignated acceptance tests. Delivered software consisted of graphical user interfaces with little functionality underneath. Customers could seldom install the software without significant assistance. Three themes emerged from the postmortem analyses we conducted across the projects. First, teams had difficulty balanc-

We started with no process, injected too much process, and then backed off to what we felt was a satisfactory balance of discipline and chaos.

September/October 2002

IEEE SOFTWARE

79

Skills that students learned in their course work did not scale well to multiple-person large projects.

80

IEEE SOFTWARE

ing the workload throughout the project. They conducted elaborate requirements statements and designs, but then discovered that they had underestimated the technical prowess necessary to implement the product within the project time frame. In most cases, their designs relied on mental models of how they thought software components— either prebuilt or organic—worked, only to discover later that those models were incomplete, too simple, or erroneous. Because they typically followed a waterfall-like lifecycle, they discarded their carefully crafted soft upstream project artifacts when building hard deliverables. Even when teams built software in iterative cycles, they tended to first concentrate on features they knew how to write and delayed working on technically difficult or complex features. Projects thus consisted of large amounts of unproductive work punctuated with short bursts of intense development. Second, skills that students learned in their course work did not scale well to multiple-person large projects. Configuration management was the most obvious stumbling block. Many teams did not have a mechanism for controlling their source code, claiming that such measures added unwanted overhead. Instead, they placed their code in a central repository, copied needed files to their local workspaces, made changes, and then replaced the files without regard to whether the original contents had changed in the interim. They depended on email and word of mouth to control inadvertent file destruction. Not only did these informal communication methods break down with erratic work schedules, but the overall scheme of continually evolving the software prevented falling back to known baselines in times of trouble. Additionally, teams reported that they did not track defects, especially in the project’s final stages, and often did not know if bug fixes were ever incorporated into the delivered product. Finally, team members’ responsibilities were not clear. Although all project participants had some technical background, few had training in team dynamics. At the project onset, teams invariably organized around an egoless model without a formal leader and few defined roles. They rationalized that this model advocated the most polite form of team government: students gen-

September/October 2002

erally felt uncomfortable about presuming a self-appointed leadership role and waited for a natural leader to emerge. They felt similarly about other team responsibilities, assuming that team members would naturally gravitate to whatever tasks felt most comfortable to them. Although some teams contained the right personalities for this to work, most teams quickly became rudderless committees with few clear lines of accountability for project activities. Team members with weak skills or shy personalities eluded productive work by shielding themselves behind ambiguous role expectations. Members with strong personalities, not wishing to receive a poor grade, compensated for weaker members by taking on additional work, thus paving the way for ill feelings. Once the personality of the team finally emerged, it was often one of cliques, contention, and miscommunication. Interestingly, during project postmortems, technical programming skills surfaced as only a minor concern. The team participants felt they were adequately equipped to write code. What they lacked were skills to deal with integrated project issues. They needed guidance—at some level of abstraction—on how to develop software. In other words, they needed a process. Process 1: MIL-STD-498 We chose to adapt MIL-STD-4984 as the first process model, abandoning an initial attempt to cobble together a software development process from scratch as too contrived. We chose this standard because it was actively used at the time, described information requirements of numerous software life-cycle activities, and was free. Working with the customer and the project supervisor, each team spent the project’s first week editing the MIL-STD-498 document specifications to meet project particulars. This period’s goal was to have each team define how it would conduct project business. In reality, this did not happen. Just as traditional teams spent precious upfront time putting together intricate but unrealistic designs, these new teams developed elaborate process descriptions that they soon jettisoned as too complex. Projects devolved to ad hoc development, but with standardized documentation. In retrospect, choosing MIL-STD-498

A Brief Process History Two disparate schools of thought regarding software development have emerged from the software industry over the past 25 years. The classical school views building software primarily as a product-driven activity, one that treats software as a “thing” with little regard to how it was developed. It professes that anyone with suitable programming skills can write software and that activities leading to a software product are free form and chaotic by nature. This school obtained and retains its outlook from programming’s early days, a time in which software development was a cottage industry that depended on expert artisans to craft software. Its hallmarks are reliance on gurus and heroes, homegrown methods and techniques, and a body of knowledge that is largely anecdotal. A contemporary school of thought appeared in the 1970s. This school drew its ideals heavily from the manufacturing sector and, as such, emphasized that the way software is built determines, to a large extent, the end product’s quality and the developers’ quality of life. This view of software as a processdriven activity recognizes the value of technical prowess central to the classical school but suggests more. It proposes that software construction is not a free-for-all but rather is best done under circumstances in which relationships between key development activities are defined explicitly. These relationships, also known as processes, identify the tasks needed to produce a software product of known quality. Most projects in industry today fall on a continuum somewhere between the classical and contemporary camps. Of particular interest is what portion of the continuum the industry emphasizes at any point in time. The industry primarily focused on the classical school in programming’s early days. Methodological processes describing the technical steps for building software appeared in the 1970s. These processes, combined with the industry’s interest in software life-cycles, pushed the philosophical focus away from pure programming classicism. This shift became more pronounced in the 1980s with the US government’s move to contain costs in embedded system software. This was manifested with the Ada movement and its attempts to marshal frameworks for common methods and tools. DOD-STD-2167 and subsequent military and industry efforts to standardize high-level life-cycle activities fueled the momentum toward the

was naïve. We thought team participants would be able to define how to conduct a project if they knew what content they needed to produce for project documents. Taking the view that documentation is a side effect of the software development effort, we thus thought our students would use the documentation templates as guides as to what activities they should perform. For seasoned software developers, this is

contemporary school during this time. The 1990s saw possibly the height of the process dialectic with the application of the ISO 9000 series to software development and the Capability Maturity Model for Software. Numerous process models appeared at this time, including MIL-STD-498 and IEEE 1074 (for a comprehensive history, see Yingxu Wang’s and Graham King’s Software Engineering Processes: Principles and Applications1). Focus on the contemporary philosophy took an interesting turn of events in the mid-1990s. Until this point, the working assumption was a more-is-better attitude in defining the way in which software is built. Standardized and de facto process models touched on virtually every development aspect. Empirical evidence showed that process discipline improved projects’ ability to contain cost, schedule, and defects;2 but this discipline came at a cost. Operating an organization at CMM Level 5, fully compliant IEEE 1074, or other so-called heavyweight process meant overhead in the process’s care and nurturing. Many developers felt encumbered by unnecessary bureaucracy in strongly process-driven efforts. This led to a grass roots backlash in the late 1990s and early 2000s against heavyweight processes to process models that were not as confining. Lightweight processes—now known as agile processes—appeared to establish equilibrium between the classical and contemporary extremes, consequently shifting the development focus to a more nonpartisan part of the philosophical continuum.3 Processes and process patterns, including Extreme Programming, Scrum, Adaptive Software Development, and Crystal, are currently embraced by the industry as ways of providing some of the contemporary school’s engineering discipline, while leaving room for the classical school’s raw creative horsepower.4 References 1. Y. Wang and G. King, Software Engineering Processes: Principles and Applications, CRC Press, Boca Raton, Fla., 2000. 2. S. McConnell, “The Business Case for Better Software Practices 2002 Keynote,” 2002, www.construx.com/BusinessCaseForSoftwarePractices-Keynote.pdf. 3. M. Fowler and J. Highsmith, “The Agile Manifesto,” Software Development, vol. 9, no. 8, Aug. 2001, pp. 28–32. 4. M. Fowler, “Put Your Process on a Diet,” Software Development, vol. 8, no. 12, Dec. 2000, pp. 32–37.

not an unrealistic expectation. However, the student teams perceived the main focus of their work to be documentation, not software. The standard outlined the content of artifacts resulting from each life-cycle stage, but it did little to help them identify and organize development tasks that would produce the artifacts. Project postmortems showed that the generic MIL-STD-498 was too abstract to achieve reasonable consisSeptember/October 2002

IEEE SOFTWARE

81

Students appreciated the high-level discipline that the TSP imposed, but they balked at the details.

82

IEEE SOFTWARE

tency among teams without a significant amount of added detail. Although teams that used MIL-STD-498 for their project refined the standard slightly for subsequent teams, it retained a document-centric aura that crippled its acceptance by the students. Process 2: IEEE 1074 We tailored the documentation templates of MIL-STD-498 to the bare minimum and then turned to IEEE 10745 to add an activity-centric flavor to the software development process. We chose this because it enumerated a development project’s activities and described the interchange of project artifacts among those activities. Thus, IEEE 1074 described project tasks and MIL-STD498 described document formats. Teams began their projects by defining their own procedures for conducting major technical development activities; that is, they defined their own Software Life Cycle Model Process and Development Process portions of IEEE 1074. We provided them with procedures for performing activities with which they had little experience, such as configuration management, product installation, project monitoring, and so on. In short, we gave them instructions on how they should carry out the standard’s Project Management, Pre-Development, Post-Development, and Integral processes. Here again, the teams were overwhelmed with the process’s weight. The portion of the process we provided was a hefty 41 pages, to which we expected the project teams to add their individual process descriptions (usually amounting to approximately 10 pages). Our document shrank to 20 pages over two years, but never seemed to be distilled to a kernel that could capture the students’ imagination and dispel their suspicions of unnecessary bureaucracy. Like the MIL-STD-498 experience, teams using IEEE 1074 developed procedures that were well intentioned but unrealistic. Team participants had been writing code in an ad hoc fashion throughout their college education; consequently, defining technical tasks that team members could carry out uniformly was unfamiliar territory. Team processes typically polarized: they were either so vague that they were of little guidance, or so legalistic that they were imprac-

September/October 2002

tical to implement and monitor. The intended effect of using IEEE 1074 was to show the teams that they could apply discipline to software development. However, because they had not experienced processes independent of structured classroom assignments, at the project’s outset, the teams focused their attention more on satisfying the process than on building a product. On average, several weeks passed before they adapted and adopted the process. Because of process overhead, IEEE 1074 teams produced the same amount of software in an entire project as the ad hoc development teams produced in several weeks, thus raising skepticism among the students about a process-oriented approach’s value. Process 3: Team Software Process Realizing that the process was encumbering the product development, we moved next in the direction of lightening the process weight by adopting the Team Software Process.6 Our students had more success with this because the process defined explicit project scripts and team roles. The TSP was out-of-the-box ready—students did not have to add their own process descriptions. Project teams understood the entire process within the project’s first days and could adjust it when needed. The time they spent on the project was no longer concentrated at product delivery time but was distributed more evenly across the project. The cyclic activities that the TSP promotes let the students build, test, and deliver software in manageable increments. Moreover, the end-of-phase reviews that it prescribes gave convenient points for the instructor and the customer to provide feedback to the team. The downside of the process was in the bookkeeping. Students overwhelmingly rejected the myriad forms that the the TSP requires, even when those forms were available electronically. They felt imposed on to follow the personal processes that the TSP prescribes—so much so that we came to doubt the veracity of process data we collected by the project’s end. Candid post mortems on these projects showed that the students appreciated the high-level discipline that the TSP imposed, but they balked at the details.

Process 4: Extreme Programming We turned to Extreme Programming7 in an effort to break out of the TSP’s heavy reliance on staff work. XP had the TSP-like advantage of being simple and understandable. It also advocated an overall development discipline in which project teams had to produce demonstrable results at regular intervals. Unlike the TSP, in which scripts describe explicit instructions on conducting various process activities, XP gives general guidelines. XP philosophy reduces software development processes to the bare essentials. It stresses a working product over elaborate documentation and measurements. It was also on the cusp of the classical school of thought; only its reliance on established, predefined commonsense activities prevented it from degenerating into chaos. XP’s emphasis on getting software working early in the project and its relaxed approach to requirements and design appealed to our project teams. It provided a suitably welcoming bridge from the extemporaneous software development of the classroom to a more disciplined mindset required for the project. Like the TSP, XP’s incremental cycles smoothed the students’ effort across the project. That we described the XP process using guidelines rather than prescriptive rules was not an obstacle. The guidelines were sufficiently detailed and intuitive enough to let the teams know what they should be doing, yet abstract enough to let the teams choose how best to carry them out. Finally, project success—which we measured by how the finished software met customers’ expectations—surpassed that of previous projects, whether they had used a process or not. Conducting the projects using XP was not without disadvantages. First, the project teams collected little statistical process data. XP’s project velocity let teams measure progress, but the metric was not detailed enough to provide any insight into process improvement. The project velocity was simple to measure and difficult to conceal, thus providing an accurate glimpse into project progress, but we had to augment it with other metrics. Second, student class schedules made XP’s pair programming activity unrealistic. Several project teams experimented with programming in virtual pairs using online collaborative tools, but they

stopped when their frustration with the tools outweighed the perceived advantages of pair programming. Third, our students underestimated XP. They mistook its informality as an invitation to seat-of-the-pants design and coding. We took special efforts to point out that although XP’s published descriptions seem casual, its mechanics of system metaphors, design refactoring, delayed optimization, and configuration management require careful attention. Lessons learned Table 1 summarizes our experiences with software processes, showing that the academic environment is not so different from the industrial one. Indeed, it mirrors in miniature industrial trends and attitudes toward software development—software developed using the classical school of thought was not meeting our customers’ needs. Our developers could not produce a consistently high-quality product by carrying out project activities extemporaneously. For us, heavyweight processes—that is, processes that were highly detailed and prescriptive— reduced project ad-libbing; however, they introduced a level of bureaucracy that interfered with product development. Ironically, like the classical school of thought, the end result was software that did not meet our customers’ needs. We had to find a suitable process weight that balanced what was being built with how it was being built. This then gave us sufficient control over projects to maintain a relatively consistent level of software quality and a relatively consistent educational experience for the students. So how can educators infuse process into projects? We have five suggestions.

Introducing processes into the classroom environment is not easier than injecting them into the workplace.

Develop a process culture Introducing processes into the classroom environment is not easier than injecting them into the workplace. It requires careful work with all project stakeholders (students, instructors, and customers) to educate them on the processes’ purpose, obtain commitments to abide by processes, identify process boundaries, and so on. Seek agility Students typically enjoy coding, but not analyzing, designing, testing, and communicating. Agile processes, such as XP, are easiest September/October 2002

IEEE SOFTWARE

83

Table 1 Process evolution over 49 capstone projects, with significant lessons learned Ad hoc

MIL-STD-498

IEEE 1074

Team Software Process

Extreme Programming

Experience level Advantage

22 projects over 10 years Rite of passage

Disadvantage

Product seldom met customer expectations Projects with actual customers Little control over project variability

5 projects over 2 years Insight into artifact content Documentation-centric outlook

10 projects over 4 years Insight into network of project activities Complexity

4 projects over 1 year Understandable process Paperwork

8 projects over 1 year Emphasis of commonsense approach Perceived informality

Documentation templates

Process framework

Knowing documentation content does not necessarily lead to knowing project activities

Overwhelming unless tailored

Cyclic development, team roles Process measurement accuracy decreases as number of measures increases

All except pair programming Refactoring, configuration management, and design simplicity are deceptively difficult

Feature adopted and carried forward Biggest lesson learned

for students to accept and perform because the processes focus on working software as the main artifact. In contrast, a heavyweight process gives the impression of ponderous bureaucracy. Matching the process weight to the students’ abilities, expectations, and tolerance is vital to a project’s success. None of the process models we constructed are bad; in fact, we used bits and pieces from each process model in the subsequent one. We found that, in our environment, lighter process models more closely matched our students’ culture. Specifically, XP was a suitable process because it was malleable enough to fit the diversity of our development efforts and the variety of our students’ skills. Develop a process infrastructure Giving the students a written process description is not enough. Instructors must also provide them with process orientation and resources they can use when they have a question. Most importantly, students must have the tools that support process activities. This includes adequate hardware and software for development, configuration management, testing, method support, and so on. We caution others attempting this that the infrastructure must support the process, not subvert it. Tools can influence the way in which students carry out process activities. An ill-conceived collection of tools can encourage actions the process does not allow, resulting in a toolprocess impedance mismatch; on the other hand, wisely selected tools can actually promote process discipline. Use processes to focus learning Few students are skilled in all the techni84

IEEE SOFTWARE

September/October 2002

cal and nontechnical activities that project work requires; consequently, instructors should develop processes that focus students’ efforts on project learning objectives. For instance, instructors might construct a project process to give only light guidance to tasks familiar to students. It might give detailed, prescriptive guidance to unfamiliar tasks that, if students invented it, would detract from their learning objectives. It might even describe a “meta” process for having the students define their own tasks for a particular segment of the project. Seek realism Like its counterpart in industry, process enactment in the classroom requires enforcement and adjustment. Instructors should expect students to know the process and adhere to its guidelines. In-project process audits are effective tools for assessing process use and perception. When a process-related problem arises, a mechanism should be in place to change the process so that the problem is not repeated in subsequent projects. Students should not be denied success because of a poor process; similarly, they should not blame the process for their lack of success.

W

e came to the hard realization that we were faced with the challenge of infusing a new technology—software processes—into the classroom. Contrary to the conventional wisdom that we could dictate to our students how we wanted them to write software, we had to carefully couple the software process to the students’ abilities, expectations, and cul-

About the Authors David A. Umphress is an associate pro-

fessor of computer science and software engineering at Auburn University. His research interests are software processes, requirements engineering, and software engineering education. He received his PhD from Texas A&M University. Contact him at the Department of Computer Science and Software Engineering, 107 Dunstan Hall, Auburn University, AL 36849; [email protected]; www.eng.auburn.edu/~umphress.

S E P T E M B E R

/

O C T O B E R

Advertiser / Product

2 0 0 2

Page Number

Codefast Inside Back Cover Hewlett Packard Back Cover John Wiley & Sons 1 SAP Labs Inside Front Cover Scientific Toolworks 11 Software Development Conference 63

T. Dean Hendrix is an associate professor of

computer science and software engineering at Auburn University. His research areas are software engineering, software visualization, and reverse engineering. He received his PhD from Auburn University. Contact him at the Department of Computer Science and Software Engineering, 107 Dunstan Hall, Auburn University, AL 36849; hendrix@eng. auburn.edu; www.eng.auburn.edu/~hendrix. James H. Cross is professor and chair of the

Department of Computer Science and Software Engineering at Auburn University. His research interests include software engineering environments, software visualization, and object-oriented methodology. He received his PhD from Texas A&M University. Contact him at the Department of Computer Science and Software Engineering, 107 Dunstan Hall, Auburn University, AL 36849; cross@eng. auburn.edu; www.eng.auburn.edu/~cross.

ture. None of the processes we used were inherently poor; we had to find the particular process that matched our culture and let us achieve the educational objectives of the project course. In the end, we found that introducing processes into our capstone project courses benefited our instructors, students, and customers. Instructors have increased visibility into projects, students have guidance on how to conduct themselves, and customers have a better engineered product. References 1.

2.

3.

4. 5.

6. 7.

S. McConnell, “The Business Case for Better Software Practices 2002 Keynote,” 2002, www.construx.com/ BusinessCaseForSoftwarePractices-Keynote.pdf. P. Bowers, “Raytheon Stands Firm on Benefits of Process Improvement,” CrossTalk, vol. 14, no. 3, Mar. 2001, pp. 9–12. M. Diaz and J. King, “How CMM Impacts Quality, Productivity, Rework, and the Bottom Line,” CrossTalk, vol. 15, no. 3, Mar. 2002, pp. 9–14. Defense Department MIL-STD-498, Software Development and Documentation, Washington, D.C., 1994. IEEE Std. 1074-1997, IEEE Standard for Developing Software Life Cycle Processes, IEEE Press, Piscataway, N.J., 1998. W. Humphrey, Introduction to the Team Software Process, Addison-Wesley, Boston, 2000. K. Beck, Extreme Programming Explained, AddisonWesley, Boston, 2000.

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

Advertising Personnel

Marion Delaney IEEE Media, Advertising Director Phone: +1 212 419 7766 Fax: +1 212 419 7589 Email: [email protected] Marian Anderson Advertising Coordinator Phone: +1 714 821 8380 Fax: +1 714 821 4010 Email: [email protected]

Sandy Brown IEEE Computer Society, Business Development Manager Phone: +1 714 821 8380 Fax: +1 714 821 4010 Email: [email protected] Debbie Sims Assistant Advertising Coordinator Phone: +1 714 821 8380 Fax: +1 714 821 4010 Email: [email protected]

Advertising Sales Representatives

Mid Atlantic (product/recruitment) Dawn Becker Phone: +1 732 772 0160 Fax: +1 732 772 0161 Email: [email protected]

New England (product) Jody Estabrook Phone: +1 978 244 0192 Fax: +1 978 244 0103 Email: [email protected]

Midwest (product) David Kovacs Phone: +1 847 705 6867 Fax: +1 847 705 6878 Email: [email protected]

New England (recruitment) Barbara Lynch Phone: +1 401 738 6237 Fax: +1 401 739 7970 Email: [email protected]

Northwest (product) John Gibbs Phone: +1 415 929 7619 Fax: +1 415 577 5198 Email: [email protected]

Midwest (recruitment) Tom Wilcoxen Phone: +1 847 498 4520 Fax: +1 847 498 5911 Email: [email protected]

Southern CA (product) Marshall Rubin Phone: +1 818 888 2407 Fax: +1 818 888 4907 Email: [email protected]

Northwest (recruitment) Mary Tonon Phone: +1 415 431 5333 Fax: +1 415 431 5335 Email: [email protected]

Southwest (product) Royce House Phone: +1 713 668 1007 Fax: +1 713 668 1176 Email: [email protected]

Southern CA (recruitment) Karin Altonaga Phone: +1 714 974 0555 Fax: + 1 714 974 6853 Email: [email protected]

Japan German Tajiri Phone: +81 42 501 9551 Fax: +81 42 501 9552 Email: [email protected] Europe Rob Walker Phone: +44 193 256 4999 Fax: +44 193 256 4998 Email: [email protected]

Southeast (product/recruitment) C. William Bentz III Email: [email protected] Gregory Maddock Email: [email protected] Sarah K. Huey Email: [email protected] Phone: +1 404 256 3800 Fax: +1 404 255 7942

September/October 2002

IEEE SOFTWARE

85

focus

educating software professionals

Teaching Distributed Multidisciplinary Software Development Lisa J. Burnell, Texas Christian University John W. Priest, University of Texas, Arlington John R. Durrett, Texas Tech University

ollaborative software development often involves people from multiple disciplines and different locations working toward a common goal in an information-rich environment. To create a more realistic distributed collaborative environment, we developed a multi-university, multidiscipline process for teaching collaborative software development. We tested this approach at Texas Christian University, the University of Texas at Arlington, and Texas Tech University. TCU

C To create a more realistic distributed collaborative environment, three Texas universities developed an innovative teaching method for collaborative software development in distributed multidisciplinary environments. 86

IEEE SOFTWARE

used this developmental process in a two-semester capstone design course, and TTU and UTA used it in traditional one-semester courses. This flexibility lets departments work in existing approved curricula. Our process exposes students to realworld issues and uses class projects from different departments that, when combined, result in a working software product. Instructors from various disciplines at the three universities teach the classes, emulating distributed software development in industry, and each class has different project tasks, responsibilities, and expertise. Students are responsible for applying their own discipline expertise, communicating the results of their efforts, and understanding feedback from other disciplines in a timely manner. A final software system and documentation is produced by the end of TCU’s second semester. UTA

September/October 2002

and TTU courses begin concurrently with TCU’s first semester, when much of the analysis, initial design, and early prototyping are done. In the second semester, new classes at UTA and TTU participate in the product development activity (depending on what courses are offered at the two campuses). The process is flexible enough to allow for such variability by modifying each school’s task assignments. Motivation Rapid technology advances and dynamic global markets can require companies to use teams of product development specialists in different locations. This change in industry paradigm— called collaborative product development or integrated product teams—has pushed for changes in university curricula, emphasizing broader topic-based design courses, 0740-7459/02/$17.00 © 2002 IEEE

team design, and communication. Accreditation Board for Engineering and Technology (ABET) rules now require programs to “demonstrate an ability to function on multidisciplinary teams, communicate effectively, and [provide] broad education … in a global and societal context” (www. abet.org). Most departments have increased the level of design content in many courses, increased communication skills, and implemented one or two semester-long capstone design courses to meet industry needs and accreditation requirements. Because each department is responsible for its own curriculum, each emphasizes the design content of that specific discipline. Thus, most courses comprise students with similar majors and backgrounds and are located at one place. However, teaching standard software engineering without considering interaction with other disciplines fails to introduce students to the total environment experience in which industry products are developed. When we demonstrate learning in a collaborative setting, students get an appreciation of different styles, priorities, and kinds of communication and develop a tolerance for differences in knowledge and skills. A small international survey conducted by Mary Last on virtual-team practices in computer science education showed interesting results (http://acad.stedwards.edu/ ~last/virtual_team_results.htm). The students surveyed reported that the difficult aspects of participating in a collaborative virtual-team project were ■

■ ■ ■ ■ ■ ■

Lack of response or response delays from counterparts (poor communication) Lack of face-to-face contact Sharing versions of design and code Interdisciplinary culture differences Arranging real-time discussion Conflicts in working patterns Lack of understanding of other disciplines

■ ■ ■ ■

Exposure to other disciplines Teamwork Flexible meeting schedules Added motivation to learn more and to learn it in depth

Students will be better prepared to understand behavioral, not just technical, issues. The lessons learned can be directly applied to improve current industry practice, so industry can benefit as well. Other surveys report that universities benefit by developing distributed, collaborative projects.1–3 Many accreditation (AACSB/ABET/CAC/ACM) and university curriculum requirements are also satisfied. The universities (and specific departments) need to provide learning experiences that meet accredidation, university, and departmental requirements. For example, at TCU the computer science department requires seniors to complete a twosemester software development sequence. The described collaborative process meets many of the internal and external requirements for course learning objectives and better prepares students for industry than conventional approaches. The university benefits because the students satisfy their requirements, and by offering better courses, universities can recruit and retain students.

The lessons learned can be directly applied to improve current industry practice, so industry can benefit as well.

Pedagogical challenges Designing effective team projects is demanding. A systematic approach is even more crucial when designing distributed, multidisciplinary, and multiuniversity projects. There are several key challenges. Simulating the industrial workplace To adequately simulate industry experiences, students should be exposed to conflicting goals and motivations (including dealing with low-performing team members), differences in skills and knowledge, lack of communication because of differing locations and perspectives, and deliverable-based performance and scheduling problems.

They also reported the major advantages: ■ ■

A sense of accomplishment Meeting students from other universities

Selecting an appropriate project structure Too much structure means that the students will not get the learning experiences of struggling with real development issues September/October 2002

IEEE SOFTWARE

87

Table 1 Major project responsibilities

Department College Course title Students Major responsibilities

Percentage of overall grade Course objective

Texas Christian University Fort Worth, Texas Computer Science Science and Engineering

University of Texas at Arlington Arlington, Texas Industrial Engineering Engineering

Software engineering and senior design (two semesters) Undergraduate Development lead: design, implementation, test, documentation

Product development Advanced Java (one semester) (one semester) Graduate and undergraduate Mostly graduate User requirements, UTA degreeServer-side processing, Web server— planning rule base, user testing database interface evaluation 30 70 Design content: Students will Server-side Java: Students will demonstrate knowledge of the product demonstrate the ability to transform development life cycle and the role of a set of product requirements into industrial engineers in product working programs using Java development. programming techniques.

40 Capstone design: Students will demonstrate computer science and software engineering skills through the development of a significant team project.

such as ambiguity, uncertainty, change, task prioritization, and trade-off analysis. Too little structure might result in students pursuing the wrong activities at the wrong time, leading to wasted effort. The instructors must find the proper balance and adjust the amount of structure (explicit directions) given to students to fit specific situations. Designing projects so that students experience real struggles is essential to learning. The instructor must anticipate these struggles and offer increasing levels of support when appropriate. Introducing development methods “just in time” is especially effective when coupled with group analysis of why some students’ chosen approaches failed. Selecting appropriate project difficulty Students learn by experience. If the project is too easy, they won’t be motivated to learn the theory and practice of software engineering—it becomes “extra work” the students feel compelled to do because of a grade, not because they see the benefit. The project requirements must be incomplete, open-ended, complex, and interweaved and necessitate designs with many subcomponents—for example, a Web-based intelligent system with database requirements. Scheduling tasks and disseminating results Meeting due dates is essential so each team can meet its schedules and instructors can synchronize lecture topics with the product task schedule. For example, stu88

IEEE SOFTWARE

September/October 2002

Texas Tech University Lubbock, Texas Information Systems Business Administration

dents from one university are required to evaluate the deliverables of students from another university. After the student evaluators submit their reports, the other students evaluate these evaluations. In this way, students receive peer feedback on their performance, letting them see how others are interpreting and using their work products. Final products are posted on the project Web sites that are maintained throughout the project. TCU students give final product presentations at the end of the second semester; these are open to all universities and local industry. Videotapes of these presentations are made available to all interested parties. Defining and coordinating faculty responsibilities Communication skills are important for any interdisciplinary team effort. Skill, experience, goals, and other differences among the students tend to cause “territorial” conflicts and shrink communication channels. Even though our method has traditional college-level instruction—repeated face-to-face contact with similarly interested students—faculty members must coordinate their efforts, do more planning up front, and remain flexible in task assignments and schedules. Our approach As in an industry project, you must establish responsibilities and a schedule for each team—in this case, for each class. Each university is from a different discipline (department), and has a unique set of

Table 2 Project schedule by institutions Schedule of tasks Weeks 1-3 Requirements phase

Week 4 Milestone 1 Architecture and early phase design Week 6 Week 8 Milestone 2 Design phase Week 12 Milestone 3 Test phase Week 15 Milestone 4

Texas Christian University Study and practice requirements

University of Texas at Arlington Develop requirements models,

Texas Tech University Study and practice of SQL scripts

analysis methods needed for milestone 1 Create analysis model, including prototype

benchmark existing systems

and interfacing to databases

Review analysis model (due one week after TCU submits)

Create initial entity relationship diagram

Provide UTA degree-planning rules and sample student transcripts Review design model and test 1st build

Develop model from entity relationship diagram Build database interface as servlets and Java server pages

Review design model, test 2nd build, evaluate test document

Build GUI front-end for database editing

Evaluate UTA review Design and implement 1st build (implement part of design) Design and implement 2nd build

Write project documentation and plan for 2nd semester, submit paper

responsibilities. Individual course objectives and student skills dictate the project task assignment (see Table 1). The percentage of time devoted and grade credit earned can vary by course, as long as each team can meet their overall product development responsibilities. Schedules are defined by milestones, when project artifacts are electronically disseminated (see Table 2). Each class must meet its deadlines, even if some tasks are incomplete, because the scheduled deliverables provide required input for the other classes. Deliverables for milestone 3 Each task is documented and sent electronically to the other university groups. TCU must provide the following milestone 3 documentation: ■ ■ ■

■

Project description and plan. Updated use case model. Design model (final design revision, at least for those aspects that have been implemented), including pseudocode for nontrivial algorithms. Your design section should start with a written discussion of the design, the trade-offs you made, and justification for your choices. Documented implementation. Your system might run on a single machine; your data sources, although they can still be local, must be close to final requirements.

Also for milestone 3, UTA must report on the TCU deliverables, with special attention

Restructure servlets and JSP as Web services

given to the analysis model, specified as use cases. It must also identify missing use cases or alternative scenarios, confusing or potentially misunderstood areas, and errors. TTU must provide GUI front ends for editing and adding records directly to the database. These applications are expected to validate data entered and to use the serverside applications created for milestone 2. Additional documentation requirements for all three universities include team organization descriptions, statements of work, weekly status reports, peer evaluations, and project presentations. Student project: An intelligent advisor system The initial project description, provided by the instructors, gives the general vision for the project, including a domain scenario to aid understanding. It outlines the technical motivation for this project, describing why it is challenging, along with the inherent technical issues to be considered in the system analysis and design. A portion of the project description appears in the “Virtual Intelligent Advisor” sidebar. In the analysis phase, students create the requirements model, which includes specification of use cases to capture functional requirements (see Figure 1). In the transition from analysis to early design, students prototype the GUI, design and document the database as an entity relationship diagram, and design the initial class model. Testing takes place throughout the project life cycle. FigSeptember/October 2002

IEEE SOFTWARE

89

The Virtual Intelligent Advisor The following is an excerpt from documentation given to each new team of students assigned to Intelligent Advisor, a Web-based support system that delivers advising services to students, such as degree planning, course approval, transfer analysis, and general information regarding types of degree plans, market outlook (types of jobs, location, and pay), and so on. Moving beyond the common yet marginally effective strategy of making paperbased advising materials electronically available, we seek to provide a means by which a student (or potential student) can

engage in a dialogue with an “Intelligent Advisor.” Students want to get answers quickly to specific questions, not wade through pages of online documents. For example, Elton wants to graduate as quickly as possible, subject to the realities of his specific situation. He can only take online or night classes, because he works full time during the day. In the summer, he could take a morning class, but he would prefer not to. He believes that if the classes are not too hard, he could take 15 hours a semester. If he has one or more tough or really

ure 2 shows the expected output from a representative test plan. Project and course evaluations We evaluated both the course and collaborative process regularly. Throughout each semester, student peer groups (from each participating institution) and faculty from one or more of the institutions evaluated student project deliverables. Industry advisory board members also evaluated this approach and reported that the process simulated a best-practices corporate environment. They also commented on the expected benefits that resulted from the multidisciplinary teams and communications facets of the project.

Description: The student can choose to create their degree plan manually for a selected number of semesters. The student is given a list of courses offered within the university. The student can then select courses for each semester. When finished, the system verifies the plan and reports errors. Primary Scenario: 1. Student selects User Defined Degree Planner option. 2. System retrieves information on student courses and displays them. 3. Student assigns courses to desired semesters and requests system verification. 4. System checks rules, including prerequisites. 5. System returns with approval or explanation of errors.

Figure 1. A use case capturing the functional requirements of a user-defined degree planning procedure.

90

IEEE SOFTWARE

September/October 2002

time-consuming classes, he should take no more than 9 or 12 hours. He wants to get a computer science degree, but if he can graduate much sooner, he would consider a computer information science degree. Before deciding to switch majors, he would like to better understand the differences between the two degrees—specifically, what kinds of jobs he can get, where those jobs are, and how much they pay. The education domain in general, as well as the Intelligent Advisor project, provides a framework for studying a

By evaluating each other’s project deliverables, the students saw how others were using their work products. This also revealed some problems with response lag time and unclear functionality and responsibilities. Too often, evaluations were returned too close to the next deliverable and therefore too late for students to make any major changes. In addition, TCU students were upset with UTA students’ ostensible lack of computer literacy, UTA students believed that TCU students ignored many of their requirements, and TTU students almost exclusively focused on user interface issues. All the groups were surprised that the other groups did not “know” what they meant. For example, a student commented, UTA students were unaware of our project specifications (even though they were a part of the deliverable) and spent too much time focusing on mundane details rather than system functionality. We wanted information on how our system performed, not on how the user interface looks. The biggest obstacle with having the UTA students evaluate our program was that most were unfamiliar with programming and, it seemed, computer functionality. Because our program was just a prototype, we could not give complete installation files to make it as easy as possible.

Student perceptions of the projects and the distributed collaborative process shows striking similarity to problems found in industry: most observations focused on teamwork, communication, and common goals. Comments included the following:

number of important concepts: Dynamic, complex rules vary widely across departments and universities and over time. ■ User modeling techniques that can adapt the user interface to the user’s knowledge, skills, and interaction styles are valuable. ■ We must provide services to an increasingly diverse, geographically distributed student community. ■

A primary goal of this system is main-

■

■

■

■

■

tainability; that is, it must be easy to adapt to specific and evolving requirements in and across universities. For example, it should be straightforward for a Texas Tech advisor to customize the system to fit his or her requirements, even though Texas Christian faculty have developed the system to fit its computer science requirements first. Some data and processes are department-, college-, and university-specific and dynamic (rapidly changing). The system needs to be designed to account for this reality. Specifically, the system should be able to

Product design. “It would have been best to design, agree upon, and fully understand all of the interfaces before we began work on the objects that use those interfaces. This would have allowed us to work independently sooner and have less difficulty with integration.” Product design. “Why can’t [we] design our own interfaces? The UTA requirements are way too complex, and we need to redesign them before we can build the database.” Communication. “At times it was unclear what each group was responsible for accomplishing. This was especially difficult when we began working together.” Teamwork. “We would rather design the database ourselves rather than waiting for the database implementation from TTU. Furthermore, by using our own database, we would have more control over direct creation and modification of data needed for each module. It is rather difficult to anticipate the time and quality of responses from groups at other universities. Rather than waiting on their response, we could already have worked out a solution to the problem.” Teamwork and scheduling. “TTU had to design and implement the database, enter the data, and create the server-side front ends before all the other students had to do anything and before we understood enough of what we were doing on the project technically.”

Fortunately, not all comments were negative. Overall, students generally expressed

Recommend at least some courses to take for next semester, checking for hard constraints (such as time conflicts) and preferences (such as morning classes only). ■ Check for a complete degree plan (proposed plan of courses over a number of semesters that satisfies degree requirements). ■ Generate a degree plan of major courses from the current semester until the student graduates, subject to student preferences and expected course offerings. ■

a great deal of satisfaction with the project. The initial requirements documents and use cases, along with the ongoing availability of student project results from other disciplines, made students feel that they were working in a more professional environment. For example, one student said, “I checked all the test cases in the documentation and everything worked just as it should have. This process of round-trip evaluation is highly recommended. I thought this team had by far the best prototype. It is more functional and usable.”

Figure 2. Test case output, as described by a test plan, shows what occurs when the student creates a degree plan with errors due to a course conflict. September/October 2002

IEEE SOFTWARE

91

Our experiences resulted in a number of lessons learned. By applying these lessons, we believe we can improve the student experience.

The final product quality is one measure of the project’s success. The final product was superior to those traditionally found in senior projects, especially in the GUI interface, adherence to user requirements, and documentation. Although the final system lacked some desired functionality, the overall result was positive. Recommendations Our experiences resulted in a number of lessons learned. By applying these recommendations, we believe we can improve the student experience. Scheduling Scheduling the due dates and distributing materials were major challenges and possibly the most important changes we would make to the process. Getting rapid turnaround so that evaluations are actually helpful is a challenge. On the date TCU projects were due, we posted the documentation and the prototypes on the course Web page. The UTA and TTU students then had to retrieve them, evaluate them, and send the results back to faculty at UTA, TTU, and TCU. We should have had these results posted as well to avoid delays in getting them distributed. The TCU students then had to assess the evaluations quickly to get them back to the UTA and TTU students. With major project deadlines every three to four weeks, the turnaround took too long. We made changes for the second semester that helped with scheduling and communication. We had each team establish special email accounts to communicate directly, with copies sent to the instructors. Some teams established their own chat rooms as well. This direct communication approach greatly improved interschool cooperation. Team mindset Creating a team mindset is crucial to success. In any effective team, cooperation with and adaptation to others is required. Technologically sophisticated students must adapt to and value the contributions of those less technologically oriented. Students must also be motivated to view the overall project success as the ultimate goal. Methods that let students put a face with a name, such as videoconferencing, would be beneficial. In the second semester, we held multi-

92

IEEE SOFTWARE

September/October 2002

ple “meet and greet” sessions at local restaurants for students at two of the schools. Students enjoyed the experience and demonstrated an improved attitude toward working with the other schools, which was reflected in their documentation and cooperation. Communication Many communication problems focused on the differences in students’ perceptions rather than in skills and backgrounds. Because students develop and analyze artifacts that the other schools actually use, they must have a clear understanding of the purpose of each task they are asked to perform. Instructors must describe the overall project tasks, each school’s role, and how they interrelate. Special challenges Teaching in a distributed, collaborative instructional environment requires special skills.4 We can divide these skills broadly into two categories: technical and behavioral. Instructors need basic technical skills such as the ability to communicate via email and the Web. This includes setting up email distribution lists, maintaining course and project Web sites, and posting student or instructor materials on the Internet. They also need to be familiar with the course objectives, methods, and student preparation in each participating course. The needed behavioral skills are adaptability and solidarity. Instructors must be able to adapt to changes in material and student progress in several courses at once rather than just in a single course. Frequent faculty communication, by conference calls, emails, or other means, is needed for the negotiation and replanning of class tasks and schedules. Instructors also must support one another by communicating to their own students why this collaborative process benefits all students and by discouraging any nonconstructive criticism of the other classes’ skills, knowledge, or value. Implementing the process The process was worth doing. According to our industry advisory board, the process successfully simulated industry problems with communication, teamwork, and conflicting requirements and priorities. By working through these problems, the end

result was a quality product that was easy to use and of which students could be proud.

T

eamwork and communication are real challenges for software development in both industry and universities. The process we have described builds skills through hands-on experiences for university students. Texts and lectures offer method and process descriptions and advice on their application, but students learn by actually doing software development and being exposed to real-world issues. In some cases, students best learn lessons by suffering the consequences of not applying sound software development processes and methods. It is the instructor’s job to provide the environment, support, and guidance to help students navigate the troubles they might experience.

Acknowledgments We thank the guest editors and anonymous reviewers for their comments and suggestions. We also thank our students for their hard work and feedback in implementing this process.

References 1. J.R. Ivins, “Interdisciplinary Project Work: Practice Makes Perfect?” IEEE Trans. Education, vol. 40, no. 3, Aug. 1997, pp. 179–183. 2. E.Z. Lui et al., “Web-Based Peer Review: The Learner as both Adapter and Reviewer,” IEEE Trans. Education, vol. 44, no. 3, Aug. 2001, pp. 246–251. 3. J.W. Priest, W.D. Bodensteiner, and N.K. Muir, “A Survey of Educational and Training Needs for Transition of a Product from Development to Manufacturing,” IEEE Trans. Education, vol. 37, no. 1, Feb. 1994, pp. 13–22. 4. I. McAlpine, “Collaborative Learning Online,” Distance Education, vol. 21, no.1, Jan. 2000, pp. 66–80.

About the Authors Lisa J. Burnell is an assistant professor of computer science at Texas Christian University. Her research interests are probabilistic reasoning, decision-theoretic inference, and software engineering methodologies. She has a BA in mathematics, and an MS and PhD in computer science all from The University of Texas at Arlington. You can reach her at Dept. of Computer Science, Texas Christian Univ., TCU Box 298850, Fort Worth TX 76129; [email protected].

John W. Priest is a professor of industrial and manufacturing engineering at The Univ. of Texas, Arlington. His technical interests include the product development process, design for manufacturing, and technical risk management. He has a BS in industrial engineering from Kettering Univ. and an MS and PhD in industrial engineering from The Univ. of Texas at Arlington. You can reach him at Dept. of Industrial and Manufacturing Systems Engineering, The Univ. of Texas at Arlington, Box 19017, Arlington, TX 76019; [email protected].

John R. Durrett is an assistant professor of information systems at Texas Tech University. His research and teaching interests are distributed systems design, e-learning, and network security. He received his PhD from the University of Texas, Austin, in information systems and his MBA and BA from West Texas A&M. You can reach him at Area of Information Systems & Quantitative Sciences, MS 2101, Texas Tech Univ., Lubbock, TX, 79409; [email protected].

Stay in the software game Visit IEEE Software on the Web at

IEEE

For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.

http://computer.org/software September/October 2002

IEEE SOFTWARE

93