,
Dr. Dobbs J O U R N A L
#375 AUGUST 2005
SOFTWARE TOOLS FOR THE PROFESSIONAL PROGRAMMER
http://www.ddj.com
DEVELOPMENT PLATFORMS Eclipse 3.1 NetBeans 4.1 Visual Studio 2005 STL-Compatible
Porting Compilers & Tools to 64 Bits Performance & .NET Apps Moving to .NET 2.0
Data Structures A Reusable Duff Device Michael Swaine
New Threads for Linux Introducing the VSTSEclipse Project Reverse Engineering & Binary Code
In the Apple Spotlight Jerry Pournelle on
Longhorn
C O N T E N T S
AUGUST 2005 VOLUME 30, ISSUE 8
FEATURES NetBeans 4.1 & Eclipse 3.1 14
FORUM
by Eric J. Bruno
NetBeans 4.1 and Eclipse 3.1 are at the forefront when it comes to development platforms for J2SE, J2EE, and J2ME.
Visual Studio 2005 Visualizers 24 by James Avery
“Visualizers” are Windows Forms dialogs in Visual Studio 2005 that let you create graphical views into the value of an object.
The Eclipse Modeling Framework 28 by Frank Budinsky
The Eclipse Modeling Framework helps you define models, from which many common code-generation patterns are generated.
The TMS Development Platform 34 by Alexander Frey
The TMS build system offers a simple, powerful way to do multiplatform development.
The VSTSEclipse Project 40 by Joe Sango
The VSTSEclipse project is focusing on an Eclipse plug-in for utilizing Visual Studio Team System functionality outside the VSTS framework.
Performance Diagnosis & .NET Applications 44 by Ramkumar N. Chintalapati and Sachin Ashok Wagh
The authors present a tool that lets you identify .NET-related problems and resolve bottlenecks during performance analysis.
Moving To .NET 2.0 48 by Eric Bergman-Terrell
.NET 2.0, C# 2.0, and Visual Studio 2005 include a host of new features.
Finding Binary Clones with Opstrings & Function Digests: Part II 56 by Andrew Schulman
Andrew continues his examination of reverse engineering, this month, focusing on binary code.
NPTL: The New Implementation of Threads for Linux 62 by L. Blunt Jackson
Introduced with Version 2.6 of the Linux kernel, the Native POSIX Thread Library brings full compliance to the POSIX Standard.
Porting Compilers & Tools to 64 Bits 67 by Steven Nakamoto and Michael Wolfe
Rehosting compilers and tools to 64-bit processors may not be as difficult as you think.
An STL-Compatible Hybrid of Linked List & Hash Map 69 by William Nagel
“Linked_hash” is an STL-compatible data structure based on the best of the linked-list hash-map classes.
EMBEDDED SYSTEMS A Reusable Duff Device 73 by Ralf Holly
Duff’s Device is a special kind of loop-unrolling mechanism that’s useful when performance counts.
COLUMNS Programming Paradigms 75
Chaos Manor 82
by Michael Swaine
by Jerry Pournelle
Embedded Space 78
Programmer’s Bookshelf 85
by Ed Nisley
by Jacek Sokulski
EDITORIAL 6 by Jonathan Erickson LETTERS 8 by you DR. ECCO’S OMNIHEURIST CORNER 11 by Dennis E. Shasha NEWS & VIEWS 12 by DDJ Staff OF INTEREST 87 by DDJ Staff SWAINE’S FLAMES 88 by Michael Swaine
RESOURCE CENTER As a service to our readers, source code, related files, and author guidelines are available at http:// www.ddj.com/. Letters to the editor, article proposals and submissions, and inquiries can be sent to
[email protected], faxed to 650-513-4618, or mailed to Dr. Dobb’s Journal, 2800 Campus Drive, San Mateo CA 94403. For subscription questions, call 800-456-1215 (U.S. or Canada). For all other countries, call 902-563-4753 or fax 902-563-4807. E-mail subscription questions to ddj@neodata .com or write to Dr. Dobb’s Journal, P.O. Box 56188, Boulder, CO 803226188. If you want to change the information you receive from CMP and others about products and services, go to http://www.cmp .com/feedback/permission.html or contact Customer Service at the address/number noted on this page. Back issues may be purchased for $9.00 per copy (which includes shipping and handling). For issue availability, send e-mail to
[email protected], fax to 785838-7566, or call 800-444-4881 (U.S. and Canada) or 785-8387500 (all other countries). Back issue orders must be prepaid. Please send payment to Dr. Dobb’s Journal, 4601 West 6th Street, Suite B, Lawrence, KS 66049-4189. Individual back articles may be purchased electronically at http://www.ddj.com/.
NEXT MONTH: In September, we turn to Communications and Networking, focusing on what’s going on when it comes to wireless.
DR. DOBB’S JOURNAL (ISSN 1044-789X) is published monthly by CMP Media LLC., 600 Harrison Street, San Francisco, CA 94017; 415-947-6000. Periodicals Postage Paid at San Francisco and at additional mailing offices. SUBSCRIPTION: $34.95 for 1 year; $69.90 for 2 years. International orders must be prepaid. Payment may be made via Mastercard, Visa, or American Express; or via U.S. funds drawn on a U.S. bank. Canada and Mexico: $45.00 per year. All other foreign: $70.00 per year. U.K. subscribers contact Jill Sutcliffe at Parkway Gordon 01-49-1875-386. POSTMASTER: Send address changes to Dr. Dobb’s Journal, P.O. Box 56188, Boulder, CO 80328-6188. Registered for GST as CMP Media LLC, GST #13288078, Customer #2116057, Agreement #40011901. INTERNATIONAL NEWSSTAND DISTRIBUTOR: Worldwide Media Service Inc., 30 Montgomery St., Jersey City, NJ 07302; 212-332-7100. Entire contents © 2005 CMP Media LLC. Dr. Dobb’s Journal is a registered trademark of CMP Media LLC. All rights reserved.
http://www.ddj.com
Dr. Dobb’s Journal, August 2005
3
,
Dr.Dobbs J O U R N A L
PUBLISHER Michael Goodman
SOFTWARE TOOLS FOR THE PROFESSIONAL PROGRAMMER
EDITOR-IN-CHIEF Jonathan Erickson
EDITORIAL MANAGING EDITOR Deirdre Blake MANAGING EDITOR, DIGITAL MEDIA Kevin Carlson SENIOR PRODUCTION EDITOR Monica E. Berg NEWS EDITOR Shannon Cochran ASSOCIATE EDITOR Della Wyser ART DIRECTOR Margaret A. Anderson SENIOR CONTRIBUTING EDITOR Al Stevens CONTRIBUTING EDITORS Bruce Schneier, Ray Duncan, Jack Woehr, Jon Bentley, Tim Kientzle, Gregory V. Wilson, Mark Nelson, Ed Nisley, Jerry Pournelle, Dennis E. Shasha EDITOR-AT-LARGE Michael Swaine PRODUCTION MANAGER Eve Gibson INTERNET OPERATIONS DIRECTOR Michael Calderon SENIOR WEB DEVELOPER Steve Goyette WEBMASTERS Sean Coady, Joe Lucca AUDIENCE DEVELOPMENT AUDIENCE DEVELOPMENT DIRECTOR Kevin Regan AUDIENCE DEVELOPMENT MANAGER Karina Medina AUDIENCE DEVELOPMENT ASSISTANT MANAGER Shomari Hines AUDIENCE DEVELOPMENT ASSISTANT Melani Benedetto-Valente MARKETING/ADVERTISING ASSOCIATE PUBLISHER Will Wise SENIOR MANAGERS, MEDIA PROGRAMS see page 86 Pauline Beall, Michael Beasley, Cassandra Clark, Ron Cordek, Mike Kelleher, Andrew Mintz MARKETING DIRECTOR Jessica Marty SENIOR ART DIRECTOR OF MARKETING Carey Perez DR. DOBB’S JOURNAL 2800 Campus Drive, San Mateo, CA 94403 650-513-4300. http://www.ddj.com/ CMP MEDIA LLC Gary Marshall President and CEO John Day Executive Vice President and CFO Steve Weitzner Executive Vice President and COO Jeff Patterson Executive Vice President, Corporate Sales & Marketing Leah Landro Executive Vice President, Human Resources Mike Mikos Chief Information Officer Bill Amstutz Senior Vice President, Operations Sandra Grayson Senior Vice President and General Counsel Alexandra Raine Senior Vice President, Communications Kate Spellman Senior Vice President, Corporate Marketing Mike Azzara Vice President, Group Director of Internet Business Robert Faletra President, Channel Group Vicki Masseria President, CMP Healthcare Media Philip Chapnick Vice President, Group Publisher Applied Technologies Michael Friedenberg Vice President, Group Publisher InformationWeek Media Network Paul Miller Vice President, Group Publisher Electronics Fritz Nelson Vice President, Group Publisher Network Computing Enterprise Architecture Group Peter Westerman Vice President, Group Publisher Software Development Media Joseph Braue Vice President, Director of Custom Integrated Marketing Solutions Shannon Aronson Corporate Director, Audience Development Michael Zane Corporate Director, Audience Development Marie Myers Corporate Director, Publishing Services
American Buisness Press
4
Dr. Dobb’s Journal, August 2005
Printed in the USA
http://www.ddj.com
EDITORIAL
TiVo This
D
igital video recorder technology in general and systems such as TiVo in particular have turned the world of television topsy-turvy, not to mention turning proper nouns into verbs. Don’t want to stay up to watch the late show? TiVo it. Irritated by an irritating commercial? TiVo it. Can’t get enough of Paris Hilton selling hamburgers? TiVo it. Part of the magic of DVR is that it truly ushers in the concepts of “time-shifting,” whereby you determine your viewing schedule, and “space shifting,” in which you transfer digital content onto gadgets other than those originally intended. TiVo’s Windows-based TiVoToGo program (http://www .tivo.com/4.9.19.asp), for instance, lets you transfer programs from TiVo DVRs directly to portable media players, laptops, and other devices. Other approaches to space shifting include TV on cell phones, like that provided by MobiTV (http://www.mobitv.com/). Reportedly, more than 300,000 subscribers have signed on with MobiTV to watch everything from Congressional hearings on C-SPAN to Larry, Curly, and Moe on the ToonWorld TV Classics channel. To make this happen, MobiTV has launched its MobiEnabled Developers Program, which provides sample MobiTV applications, product testing, and service preproduction (http://209.132.240.166/ developer/index.html). Assuming that your carrier network and cell phone support MobiTV services, 100-percent pure Java MIDP 2.0 client applications can be running on a cell phone near you real soon now. (For more information on Java MIDP, see http://java.sun.com/products/midp/.) In fact, as Linden deCarmo pointed out in “The OpenCable Application Platform” (DDJ, June 2004), Java is at the core of most of the emerging digital video technology and APIs. In addition to MobiTV’s platform, for instance, there’s TiVo’s Java-based Home Media Engine SDK (see “Building on TiVo,” by Arthur van Hoff and Adam Doppelt; DDJ, March 2005). Likewise, Java is at the heart of the recently updated OpenCable Application Platform Specification for DVRs (http://www.opencable.com/ specifications/), which defines a minimal profile for DVR software for digital cable receivers that have local storage. The OCAP DVR spec includes all required APIs, content and data formats, and protocols, up to the application level. Applications adhering to the OCAP DVR Profile are executed on OpenCable-compliant host devices. The OCAP DVR platform is applicable to a variety of hardware and operating systems, giving consumer electronics manufacturers flexibility in implementation. Granted, not everyone sees Java as the development platform for space shifting, DVRs, and other TVrelated initiatives. It’s a safe bet that Microsoft won’t be standardizing on Java APIs anytime soon for its Microsoft TV project (http://www.microsoft.com/tv/). While large corporations such as AMD, Intel, Microsoft, Apple, and others are supporting DVR standards and cranking out toolkits, homebrewers are discovering that DVR isn’t that hard. In his Circuit Cellar article “Build a Digital Video Recorder,” Miguel Sanchez presents a system built around Linux and a 500-MHz Pentium III computer with 64 MB of RAM and a 120-GB hard disk (http://www.circuitcellar.com/magazine/174toc.htm). Not to be outdone, Ken Sharp’s online article “Free TiVo: Build a Better DVR Out of an Old PC” takes you through all the hardware and software necessary for putting together a Windows-based DVR (http://www.makezine.com/extras/4.html). Still, DVR’ing isn’t a free ride, as even industry-leading TiVo (the noun) has had its share of ups and downs. One minute, former Federal Communications Commission chairman Michael Powell calls TiVo “God’s machine,” and the next thing you know, TiVo’s CEO is jumping ship and the company turns to serious cost cutting. Then before you can say “space-shift” three times, TiVo’s stock price jumps up on buyout rumors, but plummets when earnings (or the lack thereof) are announced. In truth, TiVo’s problems have nothing to do with technology, market conditions, or competition. What it really comes down to is “content”— you know, all those TV programs we’re supposed to be time and space shifting at will. For one thing, space-shifted viewing and mobile devices require a whole new kind of TV programming. Cecil B. DeMille-like productions won’t stand a chance on cell phone displays — they’re too grandiose and too long for constrained devices. Instead, new types of TV programs called “mobisodes” (short for “mobile episodes”) that are uniquely created for the screen and time limitations of cell phones are in the works. But mobisode producers are going to have to do better than what we’re seeing with the current crop of TV shows. Case in point: In a recent rerun of “Law & Order,” the editor-in-chief of a computer magazine is brutally murdered (http://www.nbc.com/Law_&_Order/episode_ guide/113.html). As if that wasn’t shocking enough, ungrateful coworkers of the poor fellow posthumously referred to him as a “pig,” among other less than endearing terms. If the Parents Television Council or some other likeminded organization wants to object to what’s on TV, forget about Paris Hilton’s hamburgers. There’s no better place to start than drivel that eliminates editors. TiVo that, Law & Order.
Jonathan Erickson editor-in-chief
[email protected] 6
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
LETTERS
, PO S S O BB
it, I was cleaning out some boxes from a recent move and found a 1976 Intel Data Catalog that I was going to throw away. Now I think I’ll try to sell it… Stuart Ball
[email protected] T
D
2
N CE
TS
2
Vintage or Quirky? Dear DDJ, When I was a social scientist at Princeton in the ’80s, I befriended a newly immigrated Russian who referred to his trade as “mathematics.” Jonathan Erickson’s June 2005 “Editorial” brought back some incredible skills possessed by this man. Recall that this was just before the fall of the Soviet Union. As part of his job at a Soviet university, he had to build microcomputers. The only microprocessors he could get were stolen or smuggled from the U.S. Here’s where it gets interesting: Most of the microprocessors were from Intel; most were rejects; and most came with a list of unimplemented or incorrectly implemented instructions! You know, 2+2=7. This guy had to build computers around processors with known defects! And since each had a different list of defects, each computer had to be designed anew, cleverly working around the defects. Oh, one more thing: This guy could effortlessly multiply and divide in hexadecimal — in his head! There were few calculators in his department, and none did hex. I’ve since hired about a dozen programmers, but I’ve never, ever seen skills like those again. Keep up the good work! David L. Ransen
[email protected] Dear DDJ, I recently read and enjoyed Jonathan Erickson’s June 2005 “Editorial” and add this site on CPU History to his list: http:// www.sasktelwebsite.net/jbayko/cpu.html. Actually, a column or series could emerge from a discussion of processor characteristics. I recall a BYTE magazine article on what certain processors did wrong. Perhaps a “What ever happened to the XXX processor?” article might prove interesting. Wil Blake
[email protected] Dear DDJ, I was glad to see Jonathan Erickson’s “Editorial” on vintage data catalogs in the June 2005 issue of DDJ. The day before I read 8
Jonathan responds: Thanks for your note, Stuart. I’m always happy to contribute to someone else’s clutter. Programming Language Popularity Dear DDJ, Rereading Jonathan Erickson’s February 2005 “Editorial” in which he mentions programming language popularity and the TIOBE Programming Community Index (http://www.tiobe.com/), a much more useful statistic would be the rankings of how many currently commercially available programs are written in which languages. A person looking to improve his programming skills would be ill-advised to choose x-language just because it has a high ranking, if it has no commercial program sellers. Nobody is going to pay you to write a program in some language that has all its applications, which are “free programs.” I would find it hard to believe that there are more currently commercially available programs in C than in C++, especially, since it is very difficult to write a Windows program in C, and very easy to write a Windows program in C++ using MFC. In fact, a really interesting statistic would be how many programs are available using MFC and how many are not. Phil Daley
[email protected] TIOBE’s Paul Jansen responds: Thanks for your TPC Index feedback, Phil. The index is not about job opportunities; there are other sites publishing such statistics. We only say that you can use our index to see whether your programming skills are still up-to-date (both commercially and noncommercially). Questions such as how many currently commercially available programs are written in a particular language are much more interesting than our index, but in my opinion, impossible to accomplish. Networking In the Fast Lane Dear DDJ, The “The Google Revolution” column by Jerry Pournelle (DDJ, February 2005) discussed his recent troubleshooting and upgrade of his network, following the outage of one of his network switches. In it he laments the marginal performance enhancement of installing gigabit Ethernet on his older systems. The PCI bus comes in different widths, depending on the quality and price of the motherboard. Server boards routinely come Dr. Dobb’s Journal, August 2005
with 64-bit PCI bus slots at varying speeds, and these slots are the only width capable of providing enough bandwidth to allow an add-in PCI gigabit card with true gigabit speeds. But there are other limits besides just the PCI bus. Purchasing a bargain basement gigabit switch is definitely not the way to go in testing the theoretical maximum of your network, as this could be the “real” bottleneck. Some switches and cards don’t handle port speed autonegotiation properly, and the gigabit interfaces will think they can only run at 10 or 100 Mbit. There is a real difference between a $75 gigE switch and a $1250 gigE switch. Jerry also mentions the idea of waiting for motherboards with gigabit Ethernet onboard. Again, there is no guarantee that just because a manufacturer put a gigE port on the board, they’ve provided sufficient bus bandwidth to run that port at full speed. Some other culprits can be shared IRQs, cheap chipsets, or bad driver/OS support. If you wired your LAN in the days before CAT5e/CAT6, you have no way of guaranteeing that the wire in your wall is capable of the enhanced speeds of gigE. The only true way to benchmark is with benchmark applications like Netperf, and using nothing but a CAT5e/CAT6 crossover cable between two machines with high quality, 64-bit add-in PCI cards in a 64-bit PCI slot. Some simple math shows that a 32-bit PCI bus is overwhelmed by the amount of data generated at true gigabit speeds, and your maximum will depend on how the PCI bus reacts to being flooded with more data than it can pass reliably. This explains the wild variation Jerry describes. I won’t go into details about why desktop-quality Windows is not a good platform to use for benchmarking the cards. Booting a Knoppix CD would provide a more reliable testing platform. David Backeberg Network Staff Assistant, MIT Math Dept.
[email protected] Jerry responds: Thanks for your letter, David. Since most of the readership probably uses desktop Windows systems, that’s the platform I tested things with. People who have professional-grade servers with 64-bit PCI bus slots will clearly get different results from those of us who tend to make do with old desktops as servers. Alas, most readers don’t have the IT budget that the MIT Math Department has. I have never for a moment supposed that there is no difference between a $1250 Ethernet switch and a $75 one, nor have I supposed that people who buy $1250 Ethernet switches should seek my advice on the purchases. I think I did state that we are using Cat5 cable. DDJ http://www.ddj.com
DR. ECCO’S OMNIHEURIST CORNER
Election Fraud in Verity Dennis E. Shasha
I
n a certain county having voting machines without paper trails, inspectors depend on exit polls to determine whether the voting machines have worked properly. Normally, they use statistics and the assumption of random sampling, but sometimes they want to be sure. The city of Verity is proud of its honest electorate. Two candidates, Fred and Wendy, have run against one another. There are only 100 voters. Each voter is given a unique number between 1 and 100 upon leaving the voting booth. The five pollsters record those numbers as well as the votes when they ask the voters how they voted. Each pollster manages to talk to 80 voters, and in every case, Fred beats Wendy by 42 to 38. Yet Wendy carries the city by 51 to 49. Upon hearing these results, Fred cries foul. You are brought in to investigate. Both Fred and Wendy agree about the following:
• The voters were honest with the pollsters and the pollsters reported their results honestly. • Every pollster spoke to 80 people, 42 of whom voted for Fred against only 38 for Wendy. • Between every pair of pollsters, all 100 people were interviewed. 1. How many pollsters could there be under these conditions for it to be possible that Wendy won, even if this were unlikely assuming random sampling? 2. How might the voters be divided among the pollsters? 3. So, was Fred right? Here is an open problem: Suppose we made a change, so that between every pair of pollsters, at least 96 distinct voters were interviewed, instead of all 100 people. In this case, how many pollsters could there be? DDJ Dennis is a professor of computer science at New York University. His most recent books are Dr. Ecco’s Cyberpuzzles (2002) and Puzzling Adventures (2005), both published by W. W. Norton. He can be contacted at
[email protected]. http://www.ddj.com
Dr. Ecco Solution Solution to “Treasure Arrow,” DDJ, July 2005. Let’s first solve the problem symbolically. The pole has length L, mass Mp, and the arrowhead has weight Ma. We represent the pole’s mass Mp as a point mass at the center. Now, let’s say we have five elastics including ones at the ends and at every quarter point. We know that the lengths of the bands must form a straight line because the pole is stiff. This observation (due to my colleague Alan Siegel) gives us a constraint in addition to balanced torque and balanced vertical force. Here, s1 is the stretch of the leftmost band in centimeters. ∆ is the difference from one band to the next one to the right, again in centimeters. s1+∆=s2 s2+∆=s3 s3+∆=s4 s4+∆=s5 The balance of vertical forces gives us: 5•s1+((1+2+3+ 4)•∆)=Mp+Ma The balance of torques gives us: L•Mp/2 from the pole.
Countertorques: L•(s1+(3/4)•(s1+∆)+(1/2)•(s1+2∆)+(1/4)(s1+3∆)) =L•((s1•(1+3/4+1/2+1/4))+ ∆•(3/4+1+3/4)) =L•(2.5s1+2.5∆) =2.5L(s1+∆)
So (1) Mp/2=2.5(s1+∆), from torque balance; (2) Mp+Ma=5(s1+2∆), from ver-
Dr. Dobb’s Journal, August 2005
tical balance. From (2), ((Mp + Ma)/5) – 2∆=s1. Therefore, using the torque balance: Mp/2=2.5(s1+∆) =2.5((((Mp+Ma)/5) –2∆)+∆) = 2.5((((Mp+Ma)/5) –∆)) =(2.5((Mp+Ma)/5)) –(2.5•∆) =(((Mp+Ma)/2)) –(2.5•∆)
So: 2.5•∆=Ma/2 ∆=Ma/5
Therefore, s1=Mp/5+Ma/5 – 2Ma/5= Mp/5 – Ma/5. This implies that if Mp= Ma, then s1=0. Since these two quantities are in fact equal, s1=0, so the left band is at its rest length of 1 meter, the right band is down 60 centimeters, and the arrow is pointing to a point that is 100 cm+2•60 cm=2.2 meters from the top. Further, ∆ is 20 centimeters. So, Mp=Ma=100 kilograms. Optimal Farming: Errata and Reader Improvements Alan Dragoo was the first to point out the bug in my solution to problem 1. He suggested a design where one circle would cover a central square and four smaller rectangles would cover side rectangles. Denis Birnie independently arrived at the same solution, as well as a very interesting solution for open problem three. You can find those at http:// cs.nyu.edu/cs/faculty/shasha/papers/ birniedobbssol.doc. DDJ
11
SECTION
A
MAIN NEWS
Dr. Dobb’s
News & Views
Free Pascal 2.0 Released After five years of development, Free Pascal 2.0, an open-source compiler compatible with Turbo Pascal and Delphi, has been released for Linux, FreeBSD, Mac OS X/Darwin, Mac OS classic, DOS, Win32, OS/2, Netware (libc and classic), and MorphOS (http://www.freepascal.org/). Free Pascal, which is a GPL compiler for 32- and 64-bit CPU architectures such as Intel 32/64-bit, AMD 32/64-bit, SPARC, PowerPC, and Intel ARM, comes with a cross-platform runtime library, interfaces to existing libraries, and nonvisual classes in the Free Component Library. Maryland Opts for Computer Recycling Under the terms of a new state law, computer manufacturers who do business in Maryland must begin paying an annual $5000 fee to offer free computer take-back programs to help recycle the state’s 60,000 tons of annual electronic waste. For the time being, the law only targets computers, but televisions may be included in the future. Annually, U.S. consumers discard approximately 50 million computers and 130 million cell phones. Other states that have adopted high-tech recycling programs include California and Maine, both of which take different approaches. California requires that consumers foot the recycling bill by paying a $6 to $10 disposal fee on every computer and TV purchased. Maine requires manufacturers to pay the complete cost of recycling computers or TVs. For more information, see http://www .marylandrecyclers.org/hb575.htm. µC++ Debuts µC++, a C++ extension that supports concurrent and real-time programming, has been released. Pronounced “micro-seeplus-plus,” µC++ 5.1.0 extends C++ with new constructs by providing advanced control-flow including lightweight concurrency on shared-memory uni- and multiprocessor computers running UNIX and Linux. µC++ (available at http://plg.uwaterloo .ca/~usystem/uC++.html) accomplishes this by providing: “coroutines,” which have independent execution states; “tasks,” which have their own threads; and “monitors” that allow for safe communication among tasks. These new classes can take part in inheritance, overloading, and templates, just like other classes. µC++ is implemented as a translator, µ++, that reads a program con12
taining the extensions and transforms each extension into one or more C++ statements, which are then compiled by an appropriate C++ compiler (currently only GNU C++) and linked with the µC++ concurrent runtime library. This release of µC++ includes support for GCC 4.0.0 and the Intel C++ 8.1 compiler, and AMD64 and Intel EM64T processors in 64-bit mode. SOA: Behind the Buzzword The OASIS Standards group has formed a committee to define exactly what constitutes a service-oriented architecture: Formally dubbed the “OASIS SOA Reference Model (SOA-RM) Technical Committee, the group will be responsible for drafting a reference model that defines the functional components of service-oriented architectures and their relationships; see the committee’s web site at http://www.oasis-open .org/committees/tc_home.php?wg_abbrev =soa-rm. The committee has 47 members, including representatives from Infosys, Boeing, Unisys, and the U.S. Department of Homeland Security. Avalon, Indigo, InfoCard Previewed Microsoft has released prebeta versions of two key pieces of its development platform for Longhorn — the Avalon presentation subsystem and the Indigo messaging framework. The “Beta1 Release Candidates” support Visual Studio 2005 Beta2 and the .NET Framework 2.0 Beta 2. Avalon, Longhorn’s 3D UI layer, includes a display engine and a managed- code framework incorporating XAML markup. New in this release is support for online video and for Microsoft’s Metro technology, an XML-based alternative to PDF. Indigo, Longhorn’s connectivity piece, is intended to support service-oriented and distributed architectures. For more information, see http://www.microsoft.com/ downloads/details.aspx?familyid=b789bc8d -4f25-4823-b6aa-c5edf432d0c1&displaylang=en. MATLAB Contest Winners Announced Congratulations to Tim Vaughan, grand prize winner of the 10th MATLAB Programming Contest sponsored by The MathWorks (http://www.mathworks.com/ contest/ants/winners.html). The problem Tim tackled involved a sandbox in which there are ants, sugar cubes, anthills, and rocks. The challenge was to write a control program used by each ant, taking into Dr. Dobb’s Journal, August 2005
DR. DOBB’S JOURNAL August 1, 2005
consideration limitations such as size and distance. Other prize contest winners include Jack Snoeyink, Hannes Naudé, Cobus Potgieter, Jan Langer, Niilo Sirola, and Timothy Alderson. Harmony: An Open-Source Java? The Apache Foundation has launched a new development effort, dubbed “Project Harmony,” to build an open-source version of Java. Harmony has been approved as part of the Apache Incubator (http:// incubator.apache.org/), a program created in 2002 as a starting point for new Apache Foundation codebases. Harmony will provide a “clean room” implementation of J2SE 5, the latest version of the Java 2 Standard Edition. The Harmony team plans to use Sun’s TCK to test compatibility with the specification — assuming that TCK can be licensed under Sun’s academic and nonprofit scholarship program. SANS Lists New Vulnerabilities The SANS Institute has issued its most recent lists of the top 20 security holes in Windows and UNIX systems, with a supplementary advisory highlighting recently discovered vulnerabilities (http://www .sans.org/top20/). The institute identified holes in Internet Explorer, ActiveX, PNG file processing, cursor and icon handling, the Windows License Logging Service, and the Microsoft Server Message Block (SMB) as the most severe of new Windows vulnerabilities in 2005; web servers and services pose the most critical ongoing threat. Meanwhile, UNIX remains plagued by holes in BIND, SNMP, and SSL. Mac OS X also made an appearance on the list due to buffer overflows in the cross-platform RealOne Player. Language Challenges The Python Challenge is a set of riddles designed to provide an entertaining way to get involved with the Python programming language by exploring Python’s module library. Among other things, the Python Challenge (http:// www.pythonchallenge.com/) demonstrates the power of Python features such as batteries. Likewise, the Ruby Quiz (http://www.rubyquiz.com/) is a weekly programming challenge for Ruby programmers with similar goals — to think, learn, and have a good time. http://www.ddj.com
PROGRAMMER’S TOOLCHEST
NetBeans 4.1 & Eclipse 3.1 Development platforms for J2SE, J2EE, and J2ME ERIC J. BRUNO
W
hen I began my career as a software developer, the modern concept of an integrated development environment (IDE) was yet to be defined. Instead, developers would simply use their favorite editor and compile code from the command line. I remember how much I looked forward to getting a cup of coffee as I started the compiler, knowing how long it would take. When the compiler was finished, I would go through any errors that were listed, open the appropriate source module, and locate the offending line of code. This was a luxury, veteran developers told me, compared to punch cards or submitting code to be compiled as overnight batch jobs. Everything changed with Microsoft’s Visual C++ 1.0. There were other IDEs available for Windows development (Borland and Symantec both offered them), but Visual C++ quickly took the lead. Over time, Microsoft set the standard for modern IDE features with Visual Studio, which included tools for building C, C++, and Visual Basic applications for Windows. With Visual J++, Microsoft even extended its support to Java. Alas, Visual J++ was abandoned, never to be extended to support Java past Version 1.1. For me, this put Java development back into the “dark ages” of using a plain old editor and command line. Quickly, vendors such as Borland and Symantec offered Java IDEs, with JBuilder and Visual Café, respectively. However, these IDEs felt clunky because they placed their own abstractions on top of important aspects of Java development, such as setting the classpath and defining packages. Because of this, not to mention a high price tag and the fact that it was still difficult (if not impossible) to debug J2EE applications running inside application servers, many developers stuck with simple editors for development. Debugging often consisted of adding System.out.prinln statements in the code. Sun recognized the shortcomings of Java development tools and offered its own IDE after acquiring NetBeans Developer from Forte. Today, it’s simply called NetBeans, and Sun has made it open source. I first used NetBeans for Java development about two years ago and wasn’t entirely comfortable with it. Consequently, I began to use Eclipse. With it, I could develop and debug Java code within a full-featured IDE that didn’t feel clunky. Soon, application server vendors such as BEA offered Eclipse plug-ins, letting us debug J2EE applications as well. Eric is a consultant in New York and has worked extensively in Java and C++ developing real-time trading and financial applications. He can be contacted at
[email protected]. 14
NetBeans 4.1 Overview With the release of NetBeans 4.1 (http://www.netbeans.org/), NetBeans has become a premier Java development platform, letting you easily build, package, and deploy standalone J2SE applications, J2EE applications (EJBs), Java web services, and (its most unique feature) wireless mobile MIDP applications that run on cell phones. NetBeans features include:
“Both Eclipse and NetBeans support Java development using multiple JDK versions” • Support for both J2SE 1.4.2 and 1.5 (development and debugging). • Support for both J2EE 1.3 and 1.4 (Servlet and/or EJB development, deployment, and debugging). • Support for J2ME Wireless Toolkit 2.2, MIDP 2.0, and CLDC 1.1 (development, debugging, and emulation). • The ability to create, test, and consume web services. • Project builds that are based entirely on Apache Ant scripts. Your projects can be built outside of NetBeans using the Ant scripts it composes, or you can import your existing Ant scripts into NetBeans. • Code refactoring that lets you easily rename fields, methods, classes, interfaces, and packages, as well as add get/set methods for fields. • Code profiling that lets you check for memory leaks and CPU performance bottlenecks. • Code editing features such as code-specific fonts and colors, code formatting, automatic indent, bookmarks, tasks, code folding (expand and collapse), auto completion, macros, and block tabbing and commenting. Configurable editor settings for Java, JSP, CSS, HTML, XML, DTS, properties, and plain text files. • A classpath that can be set per project through a straightforward interface. You can simply add JAR files or folders directly, (continued on page 18)
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
(continued from page 14) or you can add libraries, which are sets of one or more JAR files that you group together in advance. • A Navigator component (part of the editor UI) that lets you quickly browse through the methods, fields, and inheritance tree for each Java source module without scrolling through the source file itself. Overall, the NetBeans interface is attractive and efficient (see Figure 1). The default layout is as you would expect, with a component to display your currently opened projects, a portion of the screen dedicated to code editing, the navigator component, and an area to display the output of builds and debug sessions. The project display does a nice job of breaking out the components of each project, such as the source files (by package), the libraries and JAR files used in the project, and any test packages you may want to include to test your code. When you’re in debug mode, there are components to display the states of variables, show the entire call stack, and view the application’s running threads. The layout is customizable, visually appealing, and wastes no space. I found the NetBeans UI to be responsive, and in many areas slightly more responsive than Eclipse. In particular, NetBeans performed better while opening/ closing projects, and while stepping through code in the debugger. The performance difference isn’t great, but noticeable. But NetBeans is technically more than just an IDE. It is an entire platform that can be used as an application runtime, elimi-
Figure 1: NetBeans IDE.
Figure 2: Eclipse UI. 18
nating the need for you to write window, menu, or other UIrelated code. The NetBeans platform is a framework that can be used to build application user interfaces, with data storage access and presentation management. When you use the full NetBeans development platform, you simply concentrate on your application’s business logic. Eclipse 3.1 and the Web Tools Platform Overview In many ways, it’s difficult to compare Eclipse to NetBeans. The overview documentation for Eclipse (http://www.eclipse.org/) describes the platform as “an open, extensible IDE for anything, and nothing in particular.” Eclipse is a platform that can be used for Java development, as well as C, C++, Smalltalk, COBOL, Perl, Python, and Fortran. There are tools that integrate UML and other design methodologies into the platform, too. However, in this article I focus on the Eclipse plug-ins and facilities for developing Java applications. When you install the appropriate plug-ins, such as those available for WebLogic (https://eclipse-plugin.projects.dev2dev.bea.com/) and Tomcat (http://jakarta.apache.org/tomcat/resources.html), along with the Eclipse Web Tools Platform (http://www.eclipse.org/ webtools/index.html), you can develop, deploy, and debug J2EE applications with Eclipse 3.1. Among Eclipse’s features are: • Support for both J2SE 1.4.2 and 1.5 (development and debugging). • Support for both J2EE 1.3 and 1.4 (Servlet and/or EJB development, deployment, and debugging). • The ability to create, test (with the Eclipse web-service validation tools plug-in), and consume web services. • Project builds that support Ant, but the Eclipse IDE is required for all software builds. • Code refactoring. • Code profiling with the appropriate plug-in, such as that available from SourceForge (http://sourceforge.net/projects/ eclipsecolorer/). • Code formatting using predefined, customizable code styles. • The normal code-editing features, such as those supported by NetBeans. • A code outline component similar to the NetBeans Navigator that lets you browse code. • Support for programming languages and environments other than Java. The Eclipse UI (Figure 2) is just as attractive and efficient as NetBeans. However, there are more options available to customize the look-and-feel in Eclipse, such as view-specific fonts, the tab layout, and whether to display text in tabs. The fact that Eclipse breaks up the display into perspectives (different screen component layouts) is both a plus and minus. For example, I find it useful to have separate perspectives for debugging and coding, but some people may find this confusing. However, if you’re developing in different languages or want to alter the display when debugging to emphasize (say, the call stack instead of the code editor), then having separate perspectives is a necessity. In terms of performance, the one area that Eclipse outperformed NetBeans was on startup time. In all cases (with and without a project open), it took noticeably less time for Eclipse to start. However, the difference was nominal, and its significance questionable in the bigger picture. My overall opinion is that both Eclipse and NetBeans can afford quicker startup times. Standard Java Development (J2SE) Both Eclipse and NetBeans support Java development using multiple JDK versions, including 1.4.2 and 5.0. With Eclipse,
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
you can add new Java runtime environments in the Preferences dialog box, available through the menu titled Windows. Within the Preferences dialog box, expand the Java entry within the tree list, and click on Installed JREs. Here, you can add JDK/JRE entries, and set one as the default for new projects. With NetBeans, you add versions of the JDK (and set a default for new projects) via the Java Platform Manager, available from the Tools menu. Because NetBeans contains substantial support for desktop and mobile Java development, you can add development platforms for both J2SE and J2ME developments via this dialog box. The process of starting a new J2SE development project in both Eclipse and NetBeans is similar. To start, choose the New Project menu item from the File menu, available in both IDEs. With NetBeans, select the project type by choosing an item listed in the Categories list. The General category is for standard desktop Java applications, or plain-old Java object (POJO) development. With Eclipse, select the Java Project option within the Wizards tree list. After you enter a project name and select a project directory, the project is opened within each IDE’s project display (see Figure 3). Both IDEs do a good job of displaying the component of projects, either by source packages or by folders that you create within each project. The NetBeans Projects display lists components mainly by their package type, while the Files display is a straightforward folder/file list. Eclipse does the same via its Package Explorer and Navigator views, respectively. Project components are broken out differently depending upon the project. For instance, a web-application project includes additional tree nodes for web pages, web services, and web configuration files. There’s also an extensive use of icons within the different parts of a project to help you quickly navigate through the various components at a glance. It all comes down to editing code — but before you actually start typing, you need to know where to start. Both IDEs offer visual code browsers that let you quickly locate classes, methods, and fields within a project without scrolling through numerous source files. In NetBeans, it is the Navigator display, while in Eclipse it is the Outline view, as in Figure 4. In both cases, class members are displayed with icons that indicate public or private access, static or final declaration, or in the case of Eclipse, whether a method contains errors or warnings. NetBeans includes an inheritance tree within the display, whereas Eclipse uses a separate component, the Hierarchy view, to accomplish this. Both IDEs have similar code-editor features, such as syntaxspecific fonts and colors, code folding, bracket and parentheses matching, and warning/error indicators. Code folding lets you collapse portions of your code to one summary line, such as import lists, comment blocks, and entire function definitions. Both IDEs represent folding with an indicator placed within the editor’s left margin, as well as with an ellipsis (“…”) in the folded code itself. Warnings and errors are indicated, in real time, as you write code within the editor’s margin. When you hover (with the mouse) over the indicator in the margin, a tooltip is displayed with the problem description. Both NetBeans and Eclipse offer solutions to the problem when you click on the margin, although I found Eclipse to be more thorough and accurate in its resolution options. To avoid errors and reduce typing, both IDEs support what is called “auto completion.” If you type a class or object name, for example, the autocompletion feature displays the appropriate method or field names that apply. As you scroll through the list of methods and fields, the associated Javadoc entry is http://www.ddj.com
displayed. You can choose a method or field from the list displayed, or you can narrow down the choices by typing the beginning letters of the method or field you’re looking for. After choosing a method, the display shows the parameter list, highlighting the next parameter with each comma you type. This functionality is almost identical for both NetBeans and Eclipse. Once you are done editing and have all syntax-related errors cleaned up, it’s time to get the bugs out by stepping through portions of your code. Both IDEs support source-level debugging for all Java projects, although Eclipse requires the appropriate plug-ins for JSP, Servlet, and EJB debugging. The plug-in you use to accomplish this is usually the one that comes with the application server you are deploying to (such as BEA WebLogic). NetBeans comes installed with built-in support for Tomcat and Sun Java System Application Server for J2EE project debugging. While this is an advantage for NetBeans users in that it eliminates set-up, Eclipse plug-ins make it possible to develop and debug with languages and platforms not supported by NetBeans (such as C++).
Figure 3: The NetBeans Projects display and the Eclipse Package Explorer categorize the components of each project in a similar way.
Figure 4: NetBeans shows the Navigator display, while Eclipse shows the Outline view.
Dr. Dobb’s Journal, August 2005
19
Both IDEs show where breakpoints are set with an indicator in the editor’s left margin. You can set absolute breakpoints (where execution stops at a line of code) or conditional breakpoints (where execution stops when a condition is met). There is support for stepping into method calls as well as over them, jumping out of methods, and “run to cursor,” where the code executes until it reaches the selected line of code. NetBeans and Eclipse both support built-in source-code version control. Eclipse supports only CVS by default upon installation, and NetBeans supports CVS, Serena PVCS, and Microsoft Visual SourceSafe (VSS). However, Eclipse also supports PVCS and VSS via plug-ins, such as those available from SourceForge (http://sourceforge.net/projects/pvcsplugin/, and http://vssplugin .sourceforge.net/). Most source-code control vendors — such as Perforce, Subversion, Mavin, and Microsoft — offer Eclipse plugins for their tools as well. To connect to a CVS repository in Eclipse, for example, you switch to the CVS Repository Exploring perspective. Once there, right-mouse click in the CVS Repositories view, and choose the New->Repository Location menu options. When the Add CVS Repository dialog appears, enter the appropriate CVS information, and click Finish. NetBeans has a similar CVS setup dialog box, accessible through the Versioning Manager, which is available from the Versioning menu. Both IDEs let you easily add new projects to the version control system you choose, or to pull an existing project from version control. Enterprise Java Development (J2EE) Enterprise Java development has long been the domain of expensive IDEs, such as Symantec’s Visual Café, Borland’s JBuilder, and Oracle’s JDeveloper. However, with open-source IDEs such as Eclipse and NetBeans supporting the development, deployment, and debugging of J2EE applications, the barriers to entry have been
20
removed. Once again, plug-ins let Eclipse work with just about any available application server. There is built-in support for Tomcat and any J2EE-compliant application server such as BEA WebLogic, JRun, and JOnAS. Additionally, there are plug-ins available from vendors or open-source projects such as SourceForge/ObjectWeb (http://forge.objectweb.org/projects/lomboz/) and Sysdeo (http://www.sysdeo.com/). The Preferences dialog is where you configure Eclipse to work with the appropriate application server. This shows that there are various application server plug-in entries, such as Tomcat and WebLogic (which is chosen). For the WebLogic plug-in, you need to enter WebLogic-specific information, the Java classpath, and the Eclipse project(s) it applies to. Each plug-in has its own set-up requirements, but the paradigm is generally the same. To add a new application server runtime environment, such as for WebLogic, click on the Add button within the Server…Installed Runtimes preferences. Next, choose from the available server types and enter the appropriate directory and classpath information. Finally, when you create a J2EE project in Eclipse, you need to tell it which application server runtime to deploy to and run within. To do this, right-mouse click on the project name and select the New menu. Next, choose the menu option Other, then choose Server Wizard from the dialog box that appears. At this point, choose from the preconfigured J2EE application server runtimes that you created previously. NetBeans comes with built-in project configurations for Tomcat as well as Sun Java System Application Server 8.1. You can add support for just about any Java application server via the Server Manager interface, available from the Tools menu. Here, you can modify the settings of existing server definitions or add new ones via the Add Server button.
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
(continued from page 20) The NetBeans Runtime display component provides a global view of the available servers, running processes, configured databases, web servers, source control definitions, and web services defined in your development environment. The display also lets you add new servers, start/stop existing servers, connect to databases, and add web services. Additionally, you can view a
Figure 5: NetBeans Enterprise Project.
Figure 6: Eclipse web-service WSDL definition tool.
Figure 7: NetBeans UI flow designer. 22
database’s tables and stored procedures, create new tables, and execute queries. This feature is useful as it allows you to stay within the NetBeans IDE even when you need to modify a data model or test a query or stored procedure. Web-Service Development Web services can be created only within an Enterprise project in NetBeans. To create one, right-mouse click on the project name, and choose the New->Web Service menu option. A dialog box appears for you to enter the service name, the package, and location. You can choose to create one from scratch or start from an existing class or WSDL definition. The new service is listed within your project under the Web Services folder. To add a new operation, right-mouse click on the web-service name and choose Add Operation. Figure 5 is a project with two web services created (HotelWS and RestaurantWS), each with its available operations. With NetBeans, making a call to a web-service operation is simple. In an Enterprise application with a Servlet, for example, you begin in the Servlet’s Java class. In the code where you wish to call into a web service, right-mouse click in the editor, and choose Web Service Client Resources->Call Web Service Operation from the popup menu. The available web services for your environment are displayed, along with their respective operations. Selecting an operation inserts the code needed to call into that web service and handle the response. Although Eclipse doesn’t make it as easy to consume a web service as NetBeans, it does have a visual web-service definition tool, called the WSDL Editor. The tool (Figure 6) lets you visually define the WSDL for your web service. You can map whole web-service definitions to messages, specify the SOAP bindings and port types, add new operations, and define new inputs and outputs. Mobile Java Development (J2ME) NetBeans has built-in support for J2ME Mobile Java application development (MIDP). Likewise, Eclipse supports J2ME development with the appropriate plug-in. Typically, you can get a plugin from a phone vendor, such as Nokia. For this article, I worked with Nokia’s Developer Suite for J2ME (http://www.eclipseplugins.info/eclipse/plugin_details.jsp?id=712). The plug-in lets you build J2ME projects within Eclipse, and comes with emulators for the Nokia Series 60, Series 90, and 6320 models. The Nokia Eclipse plug-in adds a new project type, a MIDP Project, and adds a toolbar to the IDE. With it, you can define new packages, new MIDP classes, phone deployment parameters, and emulator configurations. When debugging, Eclipse starts the appropriate emulator and allows you to step through code seamlessly. UI development, however, is not done in Eclipse, but is done through the standalone Nokia development suite installed with the plug-in. NetBeans, however, has a J2ME UI development tool built-in — the Screen Designer. The tool lets you add menu commands (such as OK, Exit, Back, and so on), lists, text boxes, alerts, gauges, images, and other UI elements appropriate for mobile application development. When you add UI elements, NetBeans automatically generates code to handle the component actions. You need to add the implementation, but all the skeleton code is created for you. With the NetBeans Flow Designer (Figure 7), you design the flow of a J2ME application’s screens based on user actions. You can visually specify exactly what happens as users choose certain commands, enter data into a form, or choose entries from a list, for example. Doing this visually lets you quickly arrange and rearrange an application’s UI behavior without touching any code. Of course, you can still get to the code by switching the editor to the Source view at the top of the editor window. You can toggle a single Java source file between the Source view, Screen Designer view, and Flow Designer tool as you see fit; actions taken in any view affect the same source module.
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
Figure 8: NetBeans smart phone emulator. NetBeans supports J2ME application debugging and comes with a built-in smart phone emulator (Figure 8). You can interact with the application as you would on the real phone. Button and menu selection, and text entry, can be performed using the PC mouse and keyboard. As with Eclipse, you can download development tools from a specific cell phone vendor, such as Nokia, that integrate with NetBeans and include your target phone’s emulator. Conclusion Comparing IDEs such as Eclipse and NetBeans is similar to comparing development languages — there is no winner. Both IDEs are competent development platforms for J2SE, J2EE, and J2ME development. Choosing between them largely comes down to taste, influence by others, and whether you develop in other languages in addition to Java. DDJ http://www.ddj.com
Dr. Dobb’s Journal, August 2005
23
Visual Studio 2005 Visualizers See what you are debugging in Visual Studio 2005 JAMES AVERY
D
ebuggers have long been a part of the development process. Ever since the first debugger, programmers have been using debuggers as a way to step through executing code, view the values of variables, and watch how the program reacts to different situations. However, most modern debuggers have failed to adapt to the increased prevalence of complex objects in modern programming languages. These debuggers simply display the object in a tree structure that can be hard to navigate and hard to understand, or simply summarize the object using its string representation in a tool tip. This becomes even more problematic when dealing with objects that store their data as XML or some type of encoded string. Trying to determine the value of these types of variables inside the debugger becomes almost impossible. One notorious example in .NET is the DataSet object. Trying to get a look at the data inside of a DataSet is often painful as you can have multiple tables and then have a plethora of rows in each of those tables. Trying to find an individual row is an exercise in browsing through numerous tables and possibly hundreds of rows. Figure 1 shows a watch window in Visual Studio.NET 2003 displaying the data inside of a single row in a DataSet. To solve this problem, Visual Studio 2005 introduces “debugger visualizers”—Windows Forms dialogs that can be used to create graphical views into the value of an James is a .NET architect and author of Visual Studio Hacks (O’Reilly & Associates, 2004). He can be contacted at javery@ infozerk.com. 24
object, and in some cases even add additional troubleshooting functionality to the debugger. While debugging, you see a small magnifying glass next to certain types. When you click on this magnifying glass, you see one or more visualizers listed in the dropdown menu to choose from, as in Figure 2. After selecting the visualizer to use, Visual Studio then displays that visualizer; the dataset visualization can be seen in Figure 3. Using the DataSet visualizer, you can easily navigate through the tables and rows inside of a DataSet and edit the data of the DataSet. It will then be persisted back to your application. The new visualizer is leaps and bounds ahead of the old tree view model of digging through rows and tables, with an excellent editing ability to boot. Microsoft is planning to release HTML, XML, and string visualizers in addition to the DataSet visualizer with the final release of Visual Studio 2005. (This article and its code are based on the Beta 2 release of Visual Studio 2005. There is a possibility that some of the details will change before the final release of Visual Studio 2005.) How It Works Visualizers run in both the application you are debugging and the debugger: This means that they must send the object you want to visualize between these two different application domains. Visualizers accomplish this by using a System.IO.Stream, meaning that any data that will be passed from the application to the visualizer must be serialized and sent through this stream. Thus, writing a visualizer for an object that is not serializable requires a little bit of extra work. The visualizer infrastructure also provides an outgoing stream to pass data back from the visualizer to the executing application, letting you create visualizers that can edit the data they are visualizing. This means that visualizers can be more than just tools used to inspect the value of an object; they can actually become editors or analyzers, as you will see in a later example. Once the data has been transferred from the executing application to the visualizer, there are virtually endless possibilities of Dr. Dobb’s Journal, August 2005
what can be done because the dialog shown to users is simply a Windows form shown in dialog mode. Just about anything that can be done in a normal Windows Forms application could be done from a visualizer.
“Visualizers are Windows Forms dialogs that can be used to create graphical views into the value of an object” Visualizer Extensibility Model While the default visualizers are an excellent new feature, the best part is that the model is completely extensible. Without too much effort, you can quickly develop a custom visualizer, either for objects in the .NET Framework or for objects in your own projects. The visualizer extensibility model allows for visualizers to be linked to multiple classes, and multiple visualizers to be linked to a single class. To illustrate, I present a simple visualizer that can be used when debugging base64 encoded strings. Normally when strings are encoded in base64, their value is difficult to view in the debugger. You would have to cut-and-paste the value to some other tool and convert the string. Updating the string would be equally hard. Both of these problems can be solved by writing a visualizer to view, edit, and save base64 strings. The goal of this visualizer is to let users view the decoded string, then edit the value and save it back to the executing application. The first step for creating a visualizer is to create a new Class Library project in Visual Studio 2005. The next step is to right click on the references folder and add a new reference to the Microsoft.VisualStudio.DebuggerVisualizers assembly, http://www.ddj.com
The saving of the object is done through the object provider using the ReplaceObject( ) method (this method can be overridden as well if needed). Listing Two is the code used for the form.
Figure 1: Visual Studio 2003 watch window. (continued from page 24) which you can find in the .NET tab of the Add References dialog. This is the assembly that contains the interface and abstract class needed to write a custom visualizer. A basic visualizer consists of a number of different parts: • The object source that Visual Studio uses to transfer data from the application to the visualizer and back. • The Windows form that displays the data. • A small debugger side class that ties these two components together. The Object Source The role of the object source is to facilitate the transfer of the object from the application being debugged through the Visual Studio infrastructure and to the visualizer form. If the object you are visualizing is serializable, then the default object source will actually do the serialization for you. If the object is not serializable, then you will need to determine what values can be passed to the visualizer and send that data; for instance, if visualizing a SqlConnection object, you might send the connection string and connection status because the entire object cannot be serialized and sent.
Figure 2: Visual Studio 2005 visualizer.
Because the string object is serializable, I actually don’t need to create a custom object provider. However, I do so anyway for the sake of completeness and to illustrate what needs to be done in situations where automatic serialization is not possible. To create the object source: 1. Add a new class to your project. 2. Add a using statement for the Microsoft.VisualStudio.DebuggerVisualizers namespace to your class. 3. Set your class to inherit from VisualizerObjectSource. 4. Override the GetData method and provide the transport method for your object; in this example, it is simply a matter of serializing the object using the binary formatter. Listing One presents the complete object source. The Form The form is where users view and interact with the object they are visualizing. Before you can add a Windows form to your Class Library project, you first need to right click on the references folder and add a reference to the Windows.Forms assembly. For this visualizer, the form needs to: 1. Accept the object source as part of its constructor; this allows the form to read/write the value of the variable through the object source. 2. Convert the string from base64 encoding to plain text. 3. Convert the plain text back to base64 and save the object.
Deploying Visualizers Before the visualizer is ready to be deployed, you must first add an assemblylevel attribute such as this: [assembly: DebuggerVisualizer(typeof(Base64StringVisualizer.Base64StringVisualizer), typeof(Base64StringVisualizer.Base64VisualizerObjectSource), Target = typeof(string), Description = "Base64 StringVisualizer")]
The first parameter specifies your debugger side class that inherits from DialogDebuggerVisualizer. The second value specifies your custom object source. The Target parameter specifies the type that this visualizer is built for, and the description parameter specifies the name for the visualizer that should be displayed in the visualizers menu. An assembly can have multiple DebuggerVisualizer attributes, so you can link a visualizer to multiple target types or include multiple visualizers in a single assembly. Once you have added the assembly level attribute, you install the assembly. Adding a new visualizer does not involve
Figure 4: First step to running the visualizer.
Figure 3: Dataset visualization. 26
The Debugger Side Class The debugger side class is the last class needed for the visualizer. This class is used by Visual Studio to actually display your custom form. To create the debugger side class, you simply add another class to your project and set it to inherit from the DialogDebuggerVisualizer class. You then need to override the Show method and inside of that method create an instance of your visualizer form passing in the object source, then lastly, call the windowService.ShowDialog( ) method passing in your form. When the user clicks on your visualizer, the debugger calls your Show method, thus creating an instance of your form and displaying it to your user. Listing Three presents the DebuggerVisualizer class.
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
registry settings or even configuration settings, you drop your assembly into one of two different files paths: • \Documents and Setting\%profile%\My Documents\Visual Studio\Visualizers. Visualizers added here will only be available for this user. (You may need to create this directory if it does not exist.) • \%vs install directory%\Common7\ Packages\Debugger\Visualizers. Visualizers added here will be available to every user on this machine. Debugging Visualizers Because the visualizer is an assembly with no entry point, it is inherently hard to directly debug. Microsoft anticipated this and made it easy to debug visualizers by including the special class VisualizerDevelopmentHost. This class can be used to directly load your visualizer without the hassle of loading and debugging
another application. To debug your debug visualizer: 1. Add a new Console Application Project to your solution and set it as the default startup project. 2. Right click on the references folder in your console application and add references to your visualizer project as well as the Microsoft.VisualStudio.DebuggerVisualizers namespace. 3. Declare an instance of the variable you want to visualize. In this example, I create a base64 encoded string. 4. Pass your local variable to the VisualizerDevelopmentHost as in Listing Four. You can now place breakpoints in your visualizer and then simply launch the debugger, and the VisualizerDevelopmentHost launches your visualizer just like it was being launched from a normal running instance of Visual Studio.
Listing One using using using using using
Listing Two System; System.Collections.Generic; System.ComponentModel; System.Data; System.Drawing; System.Text; System.Windows.Forms; Microsoft.VisualStudio.DebuggerVisualizers; System.Runtime.Serialization.Formatters.Binary;
} }
Listing Three
namespace Base64StringVisualizer { public partial class Base64StringVisualizerForm : Form { private IVisualizerObjectProvider _objectSource; public Base64StringVisualizerForm(IVisualizerObjectProvider objectSource) { _objectSource = objectSource; InitializeComponent(); } private void Form1_Load(object sender, EventArgs e) { PopulateForm(); } private void PopulateForm() { BinaryFormatter formatter = new BinaryFormatter(); string base64string = (string) formatter.Deserialize(_objectSource.GetData()); tbBase64.Text = base64string; tbPlainText.Text = base64Decode(base64string); } private string base64Decode(string base64string) { try { UnicodeEncoding encoder = new UnicodeEncoding(); string textString = encoder.GetString(Convert.FromBase64String(base64string)); return textString; } catch (Exception e) { throw new Exception("Error during decoding", e); } }
http://www.ddj.com
DDJ
private string base64Encode(string textString) { try { string base64string = Convert.ToBase64String(ConvertStringToByteArray(textString)); return base64string; } catch (Exception e) { throw new Exception("Error during encoding", e); } } public static byte[] ConvertStringToByteArray(string stringToConvert) { return (new UnicodeEncoding()).GetBytes(stringToConvert); } private void btnCancel_Click(object sender, EventArgs e) { this.Close(); } private void btnOK_Click(object sender, EventArgs e) { string base64string = base64Encode(tbPlainText.Text); _objectSource.ReplaceObject(base64string); this.Close(); }
System; System.Collections.Generic; System.Text; Microsoft.VisualStudio.DebuggerVisualizers; System.Runtime.Serialization.Formatters.Binary;
namespace Base64StringVisualizer { class Base64VisualizerObjectSource : VisualizerObjectSource { public override void GetData(object target,System.IO.Stream outgoingData) { BinaryFormatter formatter = new BinaryFormatter(); formatter.Serialize(outgoingData, target); } } }
using using using using using using using using using
Running the Visualizer To run the visualizer, you simply need to create a test project, create a base64 encoded string, and then launch the debugger. You first see the visualizer in the dropdown visualizer menu, as in Figure 4. After selecting the visualizer in the menu, you will see the running visualizer. You can view the base64 string, the decoded string, and then edit and save the decoded string value. When you save the value, it is sent back to the executing application. Visualizers make debugging complex, modern day applications easier, and will truly enhance the debugging experience in Visual Studio 2005. The extensibility model makes it easy to create your own visualizers for either framework classes or your own classes.
using using using using using
Microsoft.VisualStudio.DebuggerVisualizers; System; System.Collections.Generic; System.Text; System.Diagnostics;
[assembly: DebuggerVisualizer(typeof(Base64StringVisualizer.DebuggerSide), typeof(Base64StringVisualizer.Base64VisualizerObjectSource), Target = typeof(string), Description = "Base64 String Visualizer")] namespace Base64StringVisualizer { class DebuggerSide : DialogDebuggerVisualizer { protected override void Show(IDialogVisualizerService windowService, IVisualizerObjectProvider objectProvider) { Base64StringVisualizerForm form = new Base64StringVisualizerForm(objectProvider); windowService.ShowDialog(form); } } }
Listing Four string base64string = "This is a test of the Base64 system"; base64string = base64Encode(s); //using method from Listing 2 VisualizerDevelopmentHost visualizerHost = new VisualizerDevelopmentHost(base64string, typeof(Base64StringVisualizer)); visualizerHost.ShowVisualizer();
Dr. Dobb’s Journal, August 2005
DDJ 27
The Eclipse Modeling Framework Moving into modeldriven development FRANK BUDINSKY
T
he idea of building applications by first modeling them, then transforming these models into implementation code has been around for many years. Providing a higher level abstraction for defining software would seem to be a natural evolution. Twenty or so years ago, structured programming languages replaced assembly language, or machine code, as the most popular way to write software. About 10 years ago, object-oriented programming languages became entrenched as the most predominant languages, again raising the abstraction level. Lately, there’s been a lot of talk about model-driven development as the next higher level abstraction. Each step in the evolution of software development has been accompanied by skepticism, and model-driven development is no different. The skepticism is usually the result of overly grandiose visions and promises, opening up the visionaries to attack from the more practical types. Many programmers think that class diagrams might be helpful to document their designs, but they know that implementing complex systems by simply “transforming a picture” is a pipe dream. They know that all the expressive power of a programming language can’t be available in a model because if it was, it wouldn’t be any simpler (higher level). It would just be another programming language. That said, most programmers do recognize that generating some of the code that Frank is an engineer in IBM’s Software Group and lead author of Eclipse Modeling Framework: A Developer’s Guide (Addison-Wesley, August 2003). He can be contacted at
[email protected]. 28
they write over and over must be possible. How many copy-and-paste operations do you need to do before you start to wonder if there couldn’t be a way of specifying parameters for patterns that you want, and just have the code generated automatically? Clearly, these patterns must represent some higher level abstraction that, if only it could be specified (modeled), could enable us to write a lot less code. The problem is that the lack of simple and seamless ways to enter the parameters for these patterns has tended to limit the use of automatic code generation in mainstream development environments. So, to foster the evolution of widespread model-driven development, a low-cost way to gradually introduce it is first needed. The Eclipse Modeling Framework (EMF; http://www.eclipse.org/emf/) has emerged as a middle-ground in the modeling versus programming worlds. Integrated with the Eclipse Java development tools, EMF provides an easy way for you to define models, from which many common codegeneration patterns are generated. It leverages the fact that just about every program we write manipulates some kind of data model. It might be defined using Java, UML, XML Schema, or some other definition language. EMF aims to extract this intrinsic “model,” thereby giving programmers an easy way to get some of the benefits of modeling, without necessarily becoming a full-fledged modeler overnight. Defining the Model Have you wondered how many of the applications you write manipulate data? The answer is pretty close to 100 percent. So where is the description of the data that your program is manipulating? Is the data simply defined by the interfaces in a Java program? Maybe you have an XML Schema that defines the data? Maybe it’s a relational database (RDB) schema? Maybe you’re already a modeling enthusiast and have a UML class diagram? Whatever the case may be, you either already have a “data model,” or are about to implicitly define it in the code you’re going to write. Dr. Dobb’s Journal, August 2005
Say that you want to write a program to manage purchase orders for some store or supplier. A purchase order includes a “bill to” and “ship to” address, and a collection of (purchase) items. An item includes a product name, quantity, and a price. If you’re a Java programmer, you’d probably start by writing some Java interfaces, something like Listing One.
“EMF is a framework and toolkit that extends Eclipse’s Java Development Tooling into modeldriven development” The abstraction provided by Java itself doesn’t explicitly define anything more than simple interfaces and methods, but through naming convention and comments, these interfaces do define the model. For example, the get/set method pairs (or Bean properties, if you want to look at it that way) represent data, or attributes, in the model. The getItems( ) method indicates that the purchase order aggregates, or is associated with (references), its corresponding items. Using this model information, one could easily imagine generating an implementation of many of these methods. Another common way to describe a data model like this one is by using an XML Schema. In this case, the same model information could be provided using XML Schema complexType and element constructs; see Listing Two. Notice that XML Schema is serving two purposes here. In addition to defining the model, it’s also specifying the persistent format of the XML http://www.ddj.com
PurchaseOrder shipTo : String billTo : String
items
Item
0..*
productName : String quantity : int price : float
Figure 1: UML class diagram of a purchase order model. (continued from page 28) file, into which the data will presumably be serialized. This dual-purpose role of XML Schema can lead to complexity, especially when some of the more advanced constructs are used, but it is becoming one of the most popular ways of modeling data structures these days. A third, and the simplest, way to represent the purchase order model would be with a UML class diagram, as in Figure 1. Not surprisingly, the UML diagram is the most concise representation. As you can see, it provides all the same information as the other two forms of the model, only this time in a simple picture. Identifying an application’s data model has two big advantages: • It gives you a nice, high-level way to both communicate the design and to share data with other applications. • You can potentially generate some, if not all, of the implementation code. Today’s applications are becoming increasingly less monolithic and more integrated with other applications. The key to supporting fine-grain data integration between applications is a common understanding of the data— that is, a model — and ideally a common implementation framework. Being able to generate some of the implementation code is also the key to increased productivity and high quality and reliability of these increasingly complex integrated applications. Modeling with EMF EMF is a framework and toolkit that extends Eclipse’s Java Development Tooling into the world of model-driven development. EMF provides a metamodel (that is, a model of a model), called “Ecore,” for
Figure 2: Eclipse workbench. 30
describing EMF models. Ecore, which is essentially the class diagram subset of UML, is based on the Object Management Group’s (OMG) Meta Object Facility (MOF) specification. The canonical form of an EMF model is an XML Metadata Interchange (XMI) serialization of an Ecore model. However, one of the main strengths of EMF is its flexibility with respect to the means by which the Ecore model can be defined: • XMI; you can create an (Ecore) XMI document directly, using an XML or text editor, or using EMF’s simple tree-based sample Ecore editor. • UML; you can define the model using a commercial UML modeling tool such as Rational Rose, or using an Eclipse plug-in such as Omondo’s free EclipseUML graphical editor (available at http:// www.omondo.com/). • Java Interfaces; basic Java interfaces with a few simple (@model ) annotations can be used to describe an Ecore model. • XML Schema; an XML Schema defining the data structures for the model, can be converted to an Ecore model. • Others; EMF is a highly extensible and customizable framework/toolkit. Support for other forms of model definition is not only possible, but expected. The first approach is the most direct, but generally appeals only to XML gurus. The second choice is the most desirable if you are already using modeling tools, while the Java approach provides the benefits of modeling in a pure Java development environment like Eclipse’s JDT. The XML Schema approach is most desirable when the tool or app is intended to manipulate XML data that is already defined using an XML Schema (as with web services). Regardless of which input form is used to define an EMF model, the benefits are the same.
Figure 3: Eclipse Java Editor. Dr. Dobb’s Journal, August 2005
Given an Ecore model of the classes, or data, of the application, EMF provides a surprisingly large portion of the benefits of modeling. The mapping between an EMF (Ecore) model and Java is natural and simple for Java programmers to understand. At the same time, it’s enough to support fine-grain data integration between applications and provide a significant productivity gain resulting from code generation. From an Ecore model, EMF’s generator can create a corresponding set of Java implementation classes. Every generated EMF class extends from the framework base class, EObject, which enables the objects to integrate and work in the EMF runtime environment. EObject provides an efficient reflective API for accessing the object’s properties generically. In addition, change notification is an intrinsic property of every EObject and an adapter framework can be used to support open-ended extension to the objects. The runtime framework also manages bidirectional reference handshaking, cross-document referencing including demand-load, and arbitrary persistent forms with a default generic XMI serialization that can be used for any EMF model. EMF even provides support for dynamic models; that is, Ecore models that are created (in memory), and then instantiated without generating code. All of the benefits of the EMF runtime framework apply equally to them. In addition to the core EMF framework, another framework, EMF.Edit, extends and builds on the core, adding support for generating adapter classes that enable viewing and command-based (undoable) editing of a model, and even a basic working model editor. Generating a Model It is straightforward to use EMF to convert an XML Schema (the purchase order schema example) into a model, then generate an implementation of the model including an XML model editor for instances of the schema. I start from an Eclipse Workbench with the purchase order schema (SimplePO.xsd) in its workspace; see Figure 2. To create the EMF (Ecore) model from the schema, all you need to do is use the EMF Project Wizard to import the model: 1. Invoke File->New->Project… and select EMF Project. 2. Name the project, that is, “simplepo.” 3. Select Load from an XML Schema and then locate the file SimplePO.xsd. 4. Proceed through the remaining wizard pages, following instructions and accepting any default values. Doing this creates a new Eclipse Java project initially containing two modelrelated files with the following suffixes: http://www.ddj.com
• ecore, an XMI serialization of the Ecore model corresponding to our XML schema. • genmodel, a generator model, which is a wrapper (decorator) model of the .ecore model that provides options relevant only to the code generator (for example, the directories in which to generate the code, and choices for other user-selectable implementation patterns). At this point, you have extracted the EMF model of the application and can now display the UML equivalent of your XML Schema by opening the .ecore file using a graphical UML editor, such as Omondo’s EclipseUML. Given that the XML Schema-based model doesn’t include graphical layout information, the UML model is presented using a default layout. The purchase order model looks nice, but more complicated models may need a bit of graphical editing. To generate the model’s corresponding Java interfaces, as well as implementation code, close the .ecore editor and open the .genmodel using the EMF code generator. A popup menu in the generator lets you generate model implementation code (Generate Model Code), adapters that support editing and display (Generate Edit Code), and even a working model editor (Generate Editor Code). Generate All generates all three. Both the Ecore and the generator models (.ecore and .genmodel) are EMF models themselves. That is, Ecore is its own metamodel, as well as the metamodel for the generator model. Both models have been generated with the EMF code generator, and the EMF code generator is even an EMF-generated editor, like the one just generated for the purchase order model. Now that you’ve generated the Java interfaces and classes, you can look at the third form (Java) of the model by opening the Java files using the Eclipse Java Editor; see Figure 3. Because you’re inside the Eclipse Java Development Environment, the generated Java code has already been compiled and is ready to run. Launching a runtime workbench, you can now create and edit XML instance documents to edit purchase orders and their corresponding items. The generated editor, although simple, is surprisingly powerful, even supporting drag-and-drop and unlimited undo. After editing the purchase order, you can save the file by invoking File->Save. The serialized form will, as expected, be an XML instance document conforming to the original schema (in SimplePO.xsd); see Figure 4. The generated EMF editor provides a good example of the recommended style for EMF model editors and can be used as either a test application or as a starting http://www.ddj.com
point from which to start customizing. This is practical because of another important advantage of EMF: You can regenerate a model after changing any of the generated source code, without wiping out your changes. The EMF generator produces files that are intended to be a combination of generated pieces and hand-written pieces. You are expected to edit the generated classes to add methods and instance variables. You can always regenerate from the model as needed, and your additions will be preserved during the regeneration. Code Generation Patterns Code generating tools often generate code that isn’t particularly clean, simple, or elegant — or even intended to be looked at. It’s just meant to be a behind-the-scenes function that you simply rely on to work. EMF’s code generation, on the other hand, is expected to be viewed, extended, and even modified, so a great deal of effort went into designing clean and elegant patterns — the kind of code you would write by hand. The intention is to solve the repetitive and sometimes difficult problems that you face over and over. Among the available patterns are: Feature Access and Notification. Simple EMF features (properties) are generated using the standard JavaBean pattern. For example, the get method for the shipTo attribute of the purchase order example simply returns an instance variable like Listing Three. The corresponding set method simply sets the same variable, but it also sends a notification to any observers that may be interested in the state change; see Listing Four. Notice that to make this method more efficient when the object has no observers, the relatively expensive call to eNotify( ) is avoided by the eNotificationRequired( ) guard. More complicated patterns are generated for other types of features, especially bidirectional references where referential integrity is maintained. In all cases, however, the code is generally as efficient as possible, given the intended semantic. Bidirectional Reference Handshaking. The implementation pattern generated for bidirectional references provides a great example of the value of EMF’s code generation. Look at an example of a simple doubly linked list of purchase orders. In UML, it might look something like Figure 5. Here you define a simple 1-to-1 bidirectional relationship, which could be used to create a chain of purchase orders. The “next” and “previous” references are used to access a purchase order’s following and previous order, respectively. By defining this as a bidirectional reference, you’re saying that the “next” reference of a givDr. Dobb’s Journal, August 2005
en purchase order and the “previous” reference of its following order should always be kept in sync. If you use Java interfaces to declare this same association, you use @model annotations in the Javadoc to indicate that the two references, “next” and “previous,” are opposites (that is, reverses) of each other, rather than simply a pair of independent references; see Listing Five. Because the two references are opposites, you can set the association from either end, and the other end is automatically updated. This may even involve removing references from other objects. For example, consider the case where three purchase orders (p1, p2, and p3) are initially linked as in Figure 6(a). If you now want to change the “next” reference of p1 to be p3, instead of p2, call the setNext( ) method on p1: p1.setNext(p3);
As a result of executing this single statement, the three objects in Figure 6(b) are updated. As you can see, there is significantly more happening than simply updating the “next” reference of p1. In fact, all of this happens: • Remove p1.next pointer to p2. • Remove p2.previous pointer to p1. • Set p1.next pointer to p3. • Remove p3.previous pointer to p2. • Remove p2.next pointer to p3. • Set p3.previous pointer to p1. • Send notifications (changes to p1, p2, and p3). To do this, the generated EMF implementation of the getNext( ) method looks something like Listing Six. Notice that, in addition to adding and removing all of the required references, the notifications are queued and only dispatched at the very end when all of the objects are in their final consistent state. Although implementing something like this isn’t that hard for good programmers, the bottom line is that it isn’t trivial and can be hard to get right. By simply leveraging the higher level EMF concept of a bidirectional relationship, we get an efficient, proven, correct implementation pattern for free. Object Persistence (Proxy Resolution and Demand Load) The ability to persist, and reference other persisted model objects, is one of the most important benefits of EMF modeling; it’s the foundation for fine-grain data integration between applications. The EMF framework provides simple, yet powerful, mechanisms for managing object persistence. Many EMF models use the default XMI serialization provided by EMF to persist 31
Figure 4: Serialized form. next
PurchaseOrder
0..1 previous 0..1
Figure 5: UML diagram of a doubly linked list of purchase orders.
their models. Others implement customized serialization, allowing the model to be persisted in its natural (native) format. For example, XML Schema models can be made persistent as .xsd files, Java models as .class files, and so on. Other models, for example, mappings, are typically persisted using (the default) XMI, although specific types of mappings may be serialized differently. For example, XML Schema mappings can also be serialized in XSLT format. Regardless of the actual persistence format, the EMF framework provides generic support for resource saving and loading, including cross-resource linking with proxy resolution and demand load. An object serialized in an XMI file, for example, can reference another object serialized in another format, or possibly even stored in a relational database. EMF’s resource cross-referencing and
(a) next p1
previous
next p2
(b)
p3
previous
next p1
p3
previous p2
demand-loading mechanisms tie them together with one simple and uniform Java API. Conclusion EMF is an open- source project at Eclipse.org providing a framework and code-generation facility for building robust applications based on surprisingly simple models. Models can be defined in several different ways — Java interfaces, XML Schemas, UML — from which EMF will generate a large part of the application. The generated code is clean, efficient, and easily hand modified. You can even regenerate the model after changing the code, without wiping out your changes. EMF provides low-cost modeling for the Java mainstream by leveraging the intrinsic model in an application. With EMF, no high-level modeling tool is required. There are two fundamental benefits from using EMF. First, it results in a productivity gain by providing a nice, high-level way (UML) to communicate the design and then generate part of the implementation code. Second, EMF allows applications to integrate at a much finer granularity than is otherwise possible. The designers of EMF believe that it mixes just the right amount of modeling with programming to maximize the effectiveness of both.
Figure 6: (a) Three purchase orders that are linked; (b) Updating the three objects using setNext().
DDJ
Listing One
Listing Four
public interface PurchaseOrder { String getShipTo(); void setShipTo(String value); String getBillTo(); void setBillTo(String value); List getItems(); // List of Item }
public void setShipTo(String newShipTo) { String oldShipTo = shipTo; shipTo = newShipTo; if (eNotificationRequired()) eNotify(new ENotificationImpl(this, Notification.SET, POPackage.PURCHASE_ORDER__SHIP_TO, oldShipTo, shipTo)); }
public interface Item { String getProductName(); void setProductName(String value); int getQuantity(); void setQuantity(int value); float getPrice(); void setPrice(float value); }
Listing Two <xsd:complexType name="PurchaseOrder"> <xsd:sequence> <xsd:element name="shipTo" type="xsd:string"/> <xsd:element name="billTo" type="xsd:string"/> <xsd:element name="items" type="PO:Item" minOccurs="0" maxOccurs="unbounded"/> <xsd:complexType name="Item"> <xsd:sequence> <xsd:element name="productName" type="xsd:string"/> <xsd:element name="quantity" type="xsd:int"/> <xsd:element name="price" type="xsd:float"/>
Listing Three
Listing Five public interface PurchaseOrder { /** @model opposite="previous" */ PurchaseOrder getNext(); /** @model opposite="next" */ PurchaseOrder getPrevious(); ... }
Listing Six public void setNext(PurchaseOrder newNext) { if (newNext != next) { NotificationChain msgs = null; if (next != null) msgs = ((InternalEObject)next).eInverseRemove(this, POPackage.PURCHASE_ORDER__PREVIOUS, PurchaseOrder.class, msgs); if (newNext != null) msgs = ((InternalEObject)newNext).eInverseAdd(this, POPackage.PURCHASE_ORDER__PREVIOUS, PurchaseOrder.class, msgs); msgs = basicSetNext(newNext, msgs); if (msgs != null) msgs.dispatch(); } else if (eNotificationRequired()) eNotify(new ENotificationImpl(this, Notification.SET, POPackage.PURCHASE_ORDER__NEXT, newNext, newNext)); // touch notification }
public String getShipTo() { return shipTo; }
32
DDJ Dr. Dobb’s Journal, August 2005
http://www.ddj.com
The TMS Development Platform A multiplatform, multitarget development system ALEXANDER FREY
W
hen developing platformindependent software, you quickly reach the point where you need a development environment that reaches beyond the familiar IDEs and makefiles, offering a more universal approach. In this article, I present a simple but powerful solution called the “TV Server Makefile System” (TMS) that we use at Fast TV Server AG (where I work) to deal with development and target platforms, ranging from Windows and Linux to various embedded processors. TMS provides development teams with a universal way of building software components, integrates a common test approach, and addresses problems that arise when supporting different hardware and platforms. Together with coding rules and component templates, we use it as a common framework for developing objectoriented software in C— still the main language for embedded programming. I used Make as the build engine because it offers three advantages: It’s free, available for every platform, and all developers are at least somewhat familiar with it. While the solution I present here is designed for C/C++, it can be used for other languages as well. One of the main goals was to keep the whole system as simple and readable as possible (which Alexander is a software engineer at FAST TV SERVER AG (http://www.tv-server.de/) in Munich. He can be contacted at
[email protected]. 34
means to not use every trick Make offers and limit the necessary preparations to a minimum) so that developers can understand or modify it. A Project In TMS, a “project” is the basic element for creating software. A project contains everything that you need for building an executable or library. Usually a project is a component in a larger system that offers a certain functionality through an API. To support a test-driven software development process, a project usually produces a library to contain the API implementation but also an executable to test the library. A project can stand on its own, as it contains all files, output- directories, testscripts, documentation, and the like required for building, testing, and using the component. In short, each project consists of the subdirectories in Table 1. The output directories (bin, extlib, lib, and obj) also contain subdirectories for each target platform (such as GCC or Visual C 6), which contain directories for debug and release versions. Although this increases the total number of directories, it offers the advantage of allowing two source/object files (or in a limited way even components) to use the same name, and it also lets you copy or backup a complete project by simply copying the project’s main directory, including its subdirectories. First Steps To be set up on your system, TMS only requires: • The GNU Make tool. On Windows you can to download and install it from the MinGW open-source project (http:// sourceforge.net/project/showfiles.php? group_id=2435). On UNIX platforms, it’s usually available in the development packages. • A system variable called “TMS_BASE_ DIR,” which contains the path to your Dr. Dobb’s Journal, August 2005
working directory but with UNIX slashes (for example, TMS_BASE_DIR=c:/ work/). On Windows you can do this in the system control panel; on Linux, using bash, you can add an export statement in your .bash_profile file inside your home directory.
“TMS addresses problems that arise when supporting different platforms”
You can easily develop on Windows for Linux. You only need a Linux server that runs Samba. You can then mount your home directory on the server (create a network drive with a fixed drive letter, for example) and put all of your working directories and files there. To build and test projects on Linux, log in to the server using an ssh or telnet client. Experienced users may prefer an X server, like that offered by Hummingbird Exceed (http://www .hummingbird.com/products/nc/exceed/) or Cygwin (http://www.cygwin.com/). The makefiles The most important thing when writing makefiles is to keep them as simple as possible. In the TMS build system, the functionality is separated in a bunch of makefiles to simplify their use and avoid duplication by separating the common from the special: http://www.ddj.com
• The project makefiles (makefile.mak, for example; see Listing One). • Common settings for specific target platforms (gcc_makefile_settings.mak; see Listing Two). • Common actions for specific target platforms (gcc_makefile_create.mak; see Listing Three). • User settings for specific target platforms (gcc_makefile_user_settings.mak; see Listing Four). Each project has its own project makefile, which consists only of a few lines. Depending on the user’s target platform, it includes the so-called settings and create makefiles, which exist only once inside the $TMS_BASE_DIR/makefiles directory. They define the platform-specific compiler and linker settings, as well as the compiling and linking process. Consequently, a project makefile only needs to specify what is really unique for the project (see Listing One): • Special compiler and linker flags (CINCS, CFLAGS, LDFLAGS). • The source files that should be compiled for the library (LIBOBJECTS) and executable (APPOBJECTS). • Tests (TESTS) that should be performed when Make is called with the test target. • The name of the library (ARCHIVE) and executable (APPLICATION). • The projects/components that are used by this project (dependencies) using their include makefiles (include…). Keeping the project makefiles this simple has two important advantages: They are easy to read and modify, and they do not contain any platform- or targetspecific code. The system makefile (Listing Two) defines the compiler, linker calls, default flags, libraries, paths, and so on. The create makefile (Listing Three) provides actions such as compiling and linking files that are performed when a certain target is defined on the console. Usually you only need to create them once when adding support for new compilers. But not all settings are user independent — some are left to be defined in the so-called user makefile (see Listing Four), which is included by the settings makefile. Here you can specify, for example, build debug or release versions, or the target hardware platform. All of these settings can also be overwritten by using system variables or command-line options. Building a Project To build a project on the console in TMS, you only need to call Make for the projects makefile. The parameters you need to specify are the target compiler (called http://www.ddj.com
TMS_TARGET, which can be also defined by a system variable if you only work with a specific compiler) and the build targets, which are performed in the given order. Possible targets are: • lib, which compiles the objects in the LIBOBJECTS list and combines them in a static library. • dll, which compiles the objects in the LIBOBJECTS list and combines them in a shared library. • exe, which compiles the objects in the APPOBJECTS list and links them together with system and user libraries to create an executable. • clean, which removes all objects, libraries, and executable files. • test, which invokes the executable with the parameters in the TESTS list once for every list item (more details will follow). • dep, which builds all items in the DEPENDENCIES list with the same settings (dependencies are not built recursively). For example, to rebuild the library and executable using the GCC compiler: make -f makefile.mak TMS_TARGET= gcc clean lib exe
Building Larger Projects By Components Larger software projects are usually built out of many different libraries, classes, and APIs. The TMS environment makes it easy to build components that can be reused by higher level projects using so-called inc-makefiles (Listing Five) located in the directory $TMS_BASE_DIR/makefiles/inc. This type of makefile acts like a header file for a specific component and can be included by other projects. It adds the path to the source (the header, for instance) and library files, as well as the library itself to the higher level projects compiler and linker flags. For example, imagine the project MyProject, which uses a component named MyTimer. The only thing you need to do is add an include statement for the inc_MyTimer makefile in the MyProject makefile; for example: include $(MAKE_DIR)/inc_MyTimer.mak
When you build MyProject, it automatically links the MyTimer library. The inc-makefiles also add an entry in the DEPENDENCIES list, which is used when you call the makefile with the target dep. Then for each project in this list, Make is called again, as you can see at the end of the inc-makefile. For example, the command: make -f makefile.mak TMS_TARGET= gcc clean lib dep
Dr. Dobb’s Journal, August 2005
rebuilds the project’s library and all libraries in the DEPENDENCIES list. Remember that all components are built in place — no header or library file needs to be copied to another location. You can order and visualize the components hierarchy by using meaningful subdirectories for your projects. Therefore, you can also build layers of software that can be reused in other projects. Platform-Specific Implementations When working on multiple platforms or when you need to support different hardware, you always encounter a situation where you need to do specific implementations of a certain functionality. For example, imagine the MyTimer component needed to be implemented differently on Windows and Linux. The first solution that comes to mind is to use #ifdefs in the code, but although this is written within a second, the consequences usually are much harder to come by. The first problem with #ifdefs is that the code gets harder to read. Especially when the number of platforms or hardware versions rise, you find it more and more difficult to tell which part is used by just taking a quick look at the source. When you need to support different versions based on the same platform/compiler, the second problem is that you need to recompile the code, which usually results in recompiling all projects every time you switch between different targets and platforms — just to be sure you are not linking some older objects. And you’ll really start to hate the #ifdefs when you need to add support for a new platform or hardware version later on. Then you will need to go through all of your code looking for #ifdefs and add support for the new target. And because you modified the source code, you should compile and test the older targets as well. The only solution to these problems is to abandon #ifdefs and instead write abstraction layers for operating systems, hardware boards, and so on. In the MyTimer example, you would create two separate projects —MyTimer_win and MyTimer_linux— using the same header file MyTimer.h, but having different implementations (see Figure 1). If you are using a versioning system such as SourceSafe or CVS, you can simply do this by sharing the header file so it’s always unique. Then you only need a small including makefile inc_MyTimer.mak that works as a switch to choose the right library at link time (Listing Six). A higher level project will now only need to include the MyTimer project and won’t have to care about the implementation anymore. Using this approach, you get rid of the previously mentioned 35
make -f makefile.mak TMS_TARGET=gcc test
#ifdef problems and improve the quality and reusability of your code.
results in two shell calls: ../bin/gcc/debug/MyTimer.out 123 ABC ../bin/gcc/debug/MyTimer.out testscript1.txt
Testing Again, TMS enables a test-driven software development process. The test code is separated from the library code and is only compiled to build the executable. This executable can use command-line parameters to perform different test cases. You should also think of using a generic test framework like CUnit; we use a simple, platform-independent parser to run test scripts. Test cases can then be added to the TESTS list in the project makefile. When calling the makefile with the test target, the executable is invoked once for every item in the list. For example, if the MyTimer project had a TESTS list like the following:
Make quits if the executable does not return 0 as a result. (This is the desired behavior to stop if a test error occurs. You can disable this by using the -I option when calling make.) An advantage is that you can compile, link, and test your project in one step; you can even test all depending projects using dep test at once. Remote Systems You can also benefit from this integrated test approach when developing for remote systems that offer some kind of remote execution. You only have to put the command for running the executable on the remote machine below the test target in the create makefile. For example, consider a remote Linux system. First, you have to make sure that the executable (and other test files or
TESTS = 123\ ABC\ testscript1.txt
A call to:
MyProject
MyTimer MyTimer_win
SocketIO
MyTimer_linux
…
…
…
Figure 1: Project hierarchy using a switch-makefile to choose the right implementation of MyTimer. Subdirectory
Description
bin: doc: extinc/extlib: lib: obj: prj: scr: test:
Executables (.out / .exe) Documentation Third-party libraries Libraries (.a) Objects (.o) makefile.mak, IDE Files (e.g. .dsw/.dsp) Sources (.h / .c / .cpp / etc.) Testscripts (.txt)
Table 1: Projects and subdirectories. Listing One
shared libraries) on the host are available for the remote machine. The easiest way to do this is to mount your host working directory using NFS. Then, inside the user makefile, create two variables: • REMOTE_IP, to contain the remote IP address (for example, REMOTE_IP = 192.168.2.1). • NFS_HOME, for the mounted working directory (for example, NFS_HOME = /var/tmp/nfs). For remote execution, we use the remote shell daemon (rshd), a background process that listens on a well-known port for remote command requests (security isn’t an issue for our development purposes). Finally, inside the create makefile, the test target can look like this: $(TESTS): @ echo "Executing on $(REMOTE_IP) $(APPLICATION) with parameter(s): $@" @ rsh $(REMOTE_IP) "exportTMS_BASE_DIR= $(NFS_HOME)$(TMS_BASE_DIR); \ cd $(NFS_HOME)$(PRJDIR);$ (NFS_HOME)$(BINDIR)/ $(APPLICATION).out $@"
Now the test target executes the tests on the remote machine, where the stdout messages are shown on the host system. It offers a transparent and fast way to do remote and embedded development. Conclusion The TMS build system offers a simple, powerful way to do multiplatform development. New compilers can easily be introduced by adding new settings and create makefiles. Using abstraction layers, new hardware targets can be added without problems and have no or only minimal impact on the existing software. Tests are an integrated part of the system, improving the software quality at the component level. Finally, it lets you choose your favorite IDE and development system. DDJ
LIBOBJECTS= $(OBJDIR)/MyProject.o
# a project makefile
APPOBJECTS= $(LIBOBJECTS) \ $(OBJDIR)/testMyProject.o
# check input ifndef TMS_BASE_DIR $(error "Error: TMS_BASE_DIR not defined!") endif ifndef TMS_TARGET $(error "Error: TMS_TARGET not defined!") endif
# tests TESTS = std # targets ARCHIVE = MyProject APPLICATION = MyProject # include extern projects (dependencies) include $(MAKE_DIR)/inc_MyTimer.mak
# the directory containing this file CURRENT_DIR = $(TMS_BASE_DIR)/MyProject/prj # include common compiler flags, definitions and user settings include $(TMS_BASE_DIR)/makefiles/$(TMS_TARGET)_makefile_settings.mak # additional include directories, compiler and linker flags CINCS += CFLAGS += LDFLAGS +=
# create targets include $(TMS_BASE_DIR)/makefiles/$(TMS_TARGET)_makefile_create.mak
Listing Two # settings makefile for gcc
(continued on page 38)
# objects
36
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
(continued from page 36)
REL_BINDIR REL_LIBDIR REL_OBJDIR REL_SRCDIR REL_PRJDIR REL_TESTDIR REL_EXTINCDIR REL_EXTLIBDIR
# check input ifndef TMS_BASE_DIR $(error "TMS_BASE_DIR not defined!") endif ifndef CURRENT_DIR $(error "CURRENT_DIR not defined!") endif ifndef TMS_TARGET $(error "TMS_TARGET not defined!") endif ifneq "$(TMS_TARGET)" "gcc" $(error "TMS_TARGET not gcc!") endif # include user settings ####################### include $(TMS_BASE_DIR)/makefiles/ user_settings/$(TMS_TARGET)_makefile_user_settings.mak # check user settings ifndef DEBUG $(error "DEBUG not defined!") endif ifndef HWPLATFORM $(error "HWPLATFORM not defined!") endif ifndef OS_TYPE $(error "OS_TYPE not defined!") endif ifndef GCC $(error "GCC not defined!") endif # You normally should not need to change the following; add # additional include-directories and compiler/linker flags in your makefile ############################################################################# CC = $(GCC)/gcc AS = $(GCC)/as LD = $(GCC)/gcc AR = $(GCC)/ar CINCS = -I. -I/usr/local/include -I/usr/include ASMFLAGS = CFLAGS = -DLINUX -DHW_X86 LDFLAGS = LDPREFIX = -l LDSUFFFIX = LDPATHPREFIX= -L # input/output directories ########################## MAKE_DIR = $(TMS_BASE_DIR)/makefiles/inc
38
= = = = = = = =
../bin/$(TMS_TARGET) ../lib/$(TMS_TARGET) ../obj/$(TMS_TARGET) ../src ../prj ../test ../extinc ../extlib/$(TMS_TARGET)
ifeq "$(DEBUG)" "YES" REL_BINDIR_EXP = $(REL_BINDIR)/debug REL_LIBDIR_EXP = $(REL_LIBDIR)/debug REL_OBJDIR_EXP = $(REL_OBJDIR)/debug REL_EXTLIBDIR_EXP = $(REL_EXTLIBDIR)/debug else REL_BINDIR_EXP = $(REL_BINDIR)/release REL_LIBDIR_EXP = $(REL_LIBDIR)/release REL_OBJDIR_EXP = $(REL_OBJDIR)/release REL_EXTLIBDIR_EXP = $(REL_EXTLIBDIR)/release endif BINDIR LIBDIR SRCDIR OBJDIR PRJDIR TESTDIR EXTINCDIR EXTLIBDIR
= = = = = = = =
$(CURRENT_DIR)/$(REL_BINDIR_EXP) $(CURRENT_DIR)/$(REL_LIBDIR_EXP) $(CURRENT_DIR)/$(REL_SRCDIR) $(CURRENT_DIR)/$(REL_OBJDIR_EXP) $(CURRENT_DIR)/$(REL_PRJDIR) $(CURRENT_DIR)/$(REL_TESTDIR) $(CURRENT_DIR)/$(REL_EXTINCDIR) $(CURRENT_DIR)/$(REL_EXTLIBDIR_EXP)
CINCS += -I$(SRCDIR) LDFLAGS += $(LDPATHPREFIX)$(LIBDIR) # shared libraries dir ifeq "$(DEBUG)" "YES" SHAREDLIB_DIR = $(TMS_BASE_DIR)/sharedlibs/$(TMS_TARGET)/debug else SHAREDLIB_DIR = $(TMS_BASE_DIR)/sharedlibs/$(TMS_TARGET)/release endif LDFLAGS += $(LDPATHPREFIX)$(SHAREDLIB_DIR) # Debug/Release versions differ by compiler/linker flags ######################################################## ifeq "$(DEBUG)" "YES" CFLAGS += -Wall -g -O0 -D_DEBUG LDFLAGS += -g -lc else CFLAGS += -Wall -O3 -DNDEBUG LDFLAGS += -lc endif # Tools
Dr. Dobb’s Journal, August 2005
http://www.ddj.com
####### ifeq "$(OS_TYPE)" "Linux" CMD_COPY = cp CMD_COPY_FLAGS = CMD_DEL = rm -f CMD_COPY_FLAGS = else $(error "OS_TYPE not Linux!") endif
ifndef HWPLATFORM HWPLATFORM = Linux endif # os version of build system ifndef OS_TYPE OS_TYPE = Linux endif # location of compliler etc. ifndef GCC GCC = /usr/bin endif
Listing Three # create makefile for gcc # additional include directories CINCS += -I$(EXTINCDIR) # additional linker flags LDFLAGS += $(LDPATHPREFIX)$(EXTLIBDIR)
Listing Five # inc-makefile for MyTimer # the name of the component MYTIMER_NAME = MyTimer
# compile objects ################### $(OBJDIR)/%.o: $(SRCDIR)/%.c @ echo "Compiling C file $