This copy is registered to: Rodney Burruss
[email protected] CLASSROOMS VIRTUAL
Online Training Courses from php|architect Zend PHP Essentials Our introductory PHP course, Zend PHP Essentials, was developed for us and Zend Technologies by PHP expert Chris Shiflett, co-founder of the PHP Security Consortium. This 19-hour course provides a thorough introduction to PHP development, with particular care to "doing things right" by covering security, performance and the best development techniques. Rather than cramming as much theory as possible, PHP Essentials provides a thoroughly practical approach to learning PHP—thus ensuring that each student will be able to write good PHP code in a real-world setting by the end of the course. Zend PHP Certification Training
Zend Professional PHP Development
If you want to become a Zend Certified Engineer, this course is the best preparation tool that you'll ever find! Designed by some of the same Subject Matter Experts who also helped write the exam itself, this course covers every single topic that is part of the exam. The Zend PHP Certification Training (course) provides a complete overview of the exam, and doubles as an excellent refresher course in PHP for any developer.
This is our advanced course for the professional PHP developer. This course picks up from where PHP Essentials ends and provides a thorough, in-depth analysis of advanced features found in both PHP 4 and PHP 5, including object-oriented programming and design patterns, XML development, regular expressions, encryption, e-mail manipulation, performance management and advanced databases.
Course
Description
Start Dates
Zend PHP Essentials
• Covers PHP 4 and PHP 5 • Provides a thorough practical Every month introduction to PHP • Covers security and performance
7 Sessions 19 Hours 3 Weeks
YES
-
$769.99 US ($999.99 CAD)
Zend PHP Certification Training
• Covers every topic in the exam • Provides an excellent refresher course for PHP at all levels
Every month
7 Sessions 19 Hours 3 Weeks
YES
Zend PHP Essentials
$644.99 US ($838.99 CAD)
Every month
7 Sessions 19 Hours 3 Weeks
YES
Zend PHP Essentials
$769.99 US ($999.99 CAD)
• Covers advanced PHP 4 and PHP 5 topics
Zend Professional • Perfect for going "beyond the PHP Development basics" and learning the true
Duration
Tutoring Prerequisites
Cost
power of PHP
• All our courses are delivered entirely online using an innovative system that combines the convenience of the Internet with the unique experience of being in a real classroom. • All sessions take place in real time, and the students can interact directly with the instructor as if they were in a real classroom either via voice or text messaging. • In most cases, our system requires no software installation and works with the majority of operating systems and browsers, including Windows, Mac OS and Linux, as well as Internet Explorer, Firefox and Safari. • All courses include a generous amount of homework and in-class exercises to ensure that the students assimilate each topics thoroughly. • Tutoring is available (via e-mail) throughout the duration of the entire course. • Each class includes a complete set of recordings that the students can peruse at their leisure.
For more information, visit our website at http://www.phparch.com/phptraining or call us toll-free at (877) 630-6202 (416-630-6202 outside Canada and the U.S.)
09.2005 DEPARTMENTS
FEATURES
6 EDITORIAL The Whining Stops Here
7 WHAT’S NEW
16 Roll Your Own Database Abstraction Module by Jason Lustig
10 TIPS & TRICKS Input Filtering: Part 3 Ensuring Input Received is Input Expected by Ben Ramsey
23 An Introduction to PDO Uniform Database Access in PHP 5.x Ilia Alshanetsky
54 TEST PATTERN State of Confusion by Marcus Baker
59 PRODUCT REVIEW
37 What are Trackbacks and Why Do They Exist? by Chris Cornutt
FUDforum 2.7.1 by Peter B. MacIntyre
63 SECURITY CORNER PHP Security Audits
44 End-to-End Testing with PHP and Internet Explorer by Oz Solomon
by Chris Shiflett
68 Exit(0); Atomic Orange by Marco Tabini
Download this month’s code at: http://www.phparch.com/code/
ED ITO RIA L
php|architect
TM
The Whining Stops Here P
HP has long been attacked by those who complain who like to complain, usually about parts of the language that “don’t [quite] work properly,” or issues that have sprung up as a result of PHP’s constant evolution (but reluctance to break backwards-compatibility). How many times have you had to consult the manual to refresh your memory on the order of the needle and haystack parameters? Unfortunately, there’s no way to “fix” this particular issue, without breaking every script, in the history of PHP, that has ever used the in_array() function. Bogus complaints aside, one actually valid argument against PHP that I’ve seen, recurring amongst the pundits, is the lack of a built-in, common database access mechanism. Sure, there are a number of database abstraction packages floating around the PHP world. Some of these are even quite mature, and featurerich. Still, none have been bundled with PHP (with the exception of PEAR::DB), nor have they received the de facto PHP Core Seal of Approval. Enter PHP Data Objects (PDO), one of, if not the, first, compiled, true PHP extensions that allows uniform database access for the majority of popular database platforms. Not only is it actually a PHP extension (which generally means that the code will be fast—and PDO meets this expectation), and not a bunch of more common PHP user-land code, but it will be bundled with PHP 5.1, which should be released “Real Soon Now.” This is great news for everyone who uses PHP to communicate with a database. One of the main PDO developers, and a name you’re likely to recognize, Ilia Alshanetsky, has written an introduction to this wonderful new extension, and we’re proud to be running it in this issue. If you’re anxious to try out PDO, but aren’t so anxious as immediately upgrade to PHP 5.1 (or a release candidate), the extension has been available in PECL for a while, now, for anyone who is running at least PHP 5.0. Back to the pundits, one thing to remember in this argument is that PDO doesn’t claim to be a database abstraction layer, but a common database access interface. True database abstraction is nearly impossible to maintain. Consider database-specific SQL, such as MySQL’s NOW() versus MSSQL’s get_date() . So, PDO aptly defers this behavior to the user, and doesn’t attempt to re-write queries (for the most part—see the part of the article that discusses prepared statements and emulation). That’s why another approach, such as the one described in Jason Lustig’s piece (in this issue) would lend itself nicely to a common access interface such as PDO. Jason’s code could easily accommodate PDO, while allowing the user to specify RDBMS-specific SQL. Looks like the PHP-haters will have to find something else to whine about. In the mean time, we PHP-lovers will go about our lives, eating up new features with enthusiasm. Happy reading!
Volume IV - Issue 9 September, 2005
Publisher Marco Tabini
Editor-in-Chief Sean Coates
Editorial Team Arbi Arzoumani Peter MacIntyre Eddie Peloke
Graphics & Layout Aleksandar Ilievski
Managing Editor Emanuela Corso
News Editor Leslie Hill
[email protected] Authors Ilia Alshanetsky, Marcus Baker, Chris Cornutt, Jason Lustig, Peter B. MacIntyre, Ben Ramsey, Chris Shiflett, Oz Solomon
php|architect (ISSN 1709-7169) is published twelve times a year by Marco Tabini & Associates, Inc., P.O. Box 54526, 1771 Avenue Road, Toronto, ON M5M 4N5, Canada. Although all possible care has been placed in assuring the accuracy of the contents of this magazine, including all associated source code, listings and figures, the publisher assumes no responsibilities with regards of use of the information contained herein or in all associated material.
Contact Information: General mailbox:
[email protected] Editorial:
[email protected] Subscriptions:
[email protected] Sales & advertising:
[email protected] Technical support:
[email protected] Copyright © 2003-2005 Marco Tabini & Associates, Inc. — All Rights Reserved
September 2005
●
PHP Architect
●
www.phparch.com
What’s
?>
NEW
PHP 5.1 RC 1 php.net announces the release of PHP 5.1 RC 1. "PHP 5.1 Release Candidate 1 is now available! If all goes well, this RC will be followed by a release within a couple of weeks. Some of the key improvements of PHP 5.1 include: • PDO (PHP Data Objects) - A new native database abstraction layer providing performance, ease-of-use, and flexibility. • Significantly improved language performance mainly due to the new Zend Engine II execution architecture. • The PCRE extension has been updated to PCRE 6.2. • Many more improvements including lots of new functionality & many bug fixes, especially in regards to SOAP, streams and SPL. • See the bundled NEWS file for a more complete list of changes. Everyone is encouraged to download and test this beta, although it is not yet recommended for mission-critical production use." Get your hands on the latest release at php.net.
MySQL 4.1.14
phpGroupWare 0.9.16.008
MySQL announces the release of version 4.1.14. Some new changes include: • SHOW CHARACTER SET and INFORMATION_SCHEMA now properly report the Latin1 character set as cp1252. • MySQL Cluster: A new -P option is available for use with the ndb_mgmd client. When called with this option, ndb_mgmd prints all configuration data to stdout, then exits. • The output of perror --help now displays the --ndb option. • NDB: Improved handling of the configuration variables NoOfPagesToDiskDuringRestartACC, NoOfPagesToDiskAfterRestartACC, NoOfPagesToDiskDuringRestartTUP, and NoOfPagesToDiskAfterRestartTUP should result in noticeably faster startup times for MySQL Cluster. • Added support of WHERE clause for queries with FROM DUAL. • Added an optimization that avoids key access with NULL keys for the ref method when used in outer joins. • Added new query cache test for the embedded server to the test suite, there are now specific tests for the embedded and nonembedded servers. • Release also contains several bug fixes.
The phpGroupWare team is proud to announce their latest release, 0.9.16.008. What is phpGroupWare? phpGroupWare.org describes it as:
Grab the latest release from mysql.com.
"phpGroupWare-formerly known as webdistro-is a multi-user groupware suite written in PHP. It provides about 50 web-based applications, such as Calendar, Address Book, an advanced Projects manager, To Do List, Notes, Email, Newsgroup and Headlines Reader, a File Manager and many more applications. The calendar supports repeating events and includes alarm functions. The email system supports inline graphics and file attachments. The system as a whole supports user preferences, themes, user permissions, multi-language support and user groups. It includes modules to set up and administer the working environment. The groupware suite is based on an advanced Application Programming Interface (API)." Get more info at phpGroupWare.org.
September 2005
●
PHP Architect
●
www.phparch.com
7
What’s New?>
Check out some of the hottest new releases from PEAR.
MP3_ID 1.2.0RC2 This class offers methods for reading and writing information tags (version 1) in MP3 files.
File_Find 1.0.0 File_Find, created as a replacement for its Perl counterpart, also named File_Find, is a directory searcher, which handles, globbing, recursive directory searching, as well as a slew of other cool features.
PHPUnit 1.3.0 PHPUnit is a regression testing framework used by developers to implement unit tests in PHP. This version is to be used with PHP 4.
Mail 1.1.8 PEAR's Mail package defines an interface for implementing mailers under the PEAR hierarchy. It also provides supporting functions that are useful to multiple mailer backends. Currently supported backends include: PHP's native mail() function, sendmail, and SMTP. This package also provides a RFC822 email address list validation utility class.
DB_DataObject_FormBuilder 0.18.1 DB_DataObject_FormBuilder will aid you in rapid application development using the DB_DataObject and HTML_QuickForm packages. For a quick, but working, prototype of your application, simply model the database, run DataObject's createTable script over it, and write a script that passes one of the resulting objects to the FormBuilder class. The FormBuilder will automatically generate a simple but working HTML_QuickForm object that you can use to test your application. It also provides a processing method that will automatically detect if an insert() or update() command has to be executed after the form has been submitted. If you have set up DataObject's links.ini file correctly, it will also automatically detect if a table field is a foreign key and will populate a select box with the linked table's entries. There are many optional parameters that you can place in your DataObjects.ini or in the properties of your derived classes, and will be used to fine-tune the form generation, gradually turning the prototypes into fully-featured forms. You can take control at any stage of the process.
Net_Curl 1.2.2 Provides an OO interface to PHP's curl extension.
php|architect Releases New Design Patterns Book We're proud to announce the release of php|architect's Guide to PHP Design Patterns, the latest release in our Nanobook series. You have probably heard a lot about Design Patterns---a technique that helps you design rock-solid solutions to practical problems that programmers everywhere encounter in their day-to-day work. Even though there has been a lot of buzz, however, no-one has yet come up with a comprehensive resource on design patterns for PHP developers—until today. Author Jason E. Sweat's book php|architect's Guide to PHP Design Patterns is the first, comprehensive guide to design patterns designed specifically for the PHP developer. This book includes coverage of 16 design patterns with a specific eye to their applications in PHP when building complex web applications, both in PHP 4 and PHP 5 (where appropriate, sample code for both versions of the language is provided). For more information, http://www.phparch.com/shop_product.php?itemid=96.
September 2005
●
PHP Architect
●
www.phparch.com
8
What’s New?>
Looking for a new PHP Extension? Check out some of the lastest offerings from PECL.
pecl_http 0.12.0 pecl_http's features and functionality includes: • Building absolute URIs • RFC compliant HTTP redirects • RFC compliant HTTP date handling • Parsing of HTTP headers and messages • Caching by "Last-Modified" and/or ETag (with 'on the fly' option for ETag generation from buffered output) • Sending data/files/streams with (multiple) ranges support • Negotiating user preferred language/charset • Convenient request functions built upon libcurl • PHP 5 classes: HttpUtil, HttpResponse, HttpRequest, HttpRequestPool, HttpMessage
APC 3.0.8 APC is the Alternative PHP Cache. It was conceived of to provide a free, open, and robust framework for caching and optimizing PHP intermediate code.
ingres 1.1 This extension supports Computer Associates's Ingres Relational Database.
DTrace 1.0.2 Allows Solaris' dtrace to instrument PHP.
PHPEd 4.0 NuSphere announces the latest release of their php IDE: PHPEd 4.0. The announcement lists some of the main features of the new release as: • Advanced, efficient and highly customizable EDITOR with support for object-oriented coding. Code highlighter, user-defined shortcuts, instant syntax analysis, code insight, code templates and much more. • Sophisticated PHP DEBUGGER that can operate both locally and in the remote mode. Debugger module for the latest php version 5.0.4 is included in the package. • PHP PROFILER. PhpED profiler shows executing time for each line, function or module of the code with tenth milliseconds precision. All the bottlenecks in the code are located quickly and efficiently. • Project-wide CODE EXPLORER in PhpED IDE shows all php classes, methods, properties, functions and variables in every detail. • Enhanced project management and deployment. Support for FTPS (TLS/SSL), SFTP and WebDAV/HTTPS (SSL) protocols make deployment and data transfer secure. • Integrated MySQL, MSSQL, Oracle and UltraSQL/PostgreSQL clients. Connect to a database directly from the IDE. Browse databases, run SQL queries and work with database content without leaving the IDE. • Integrated CVS client. Review changes in old versions of a source files to track bugs while working on the same project in a team of developers. • NuSOAP Wizard. Easily build professional web services in PHP using the NuSoap library. • Enhanced integration. PhpED IDE can be easily integrated with 3rd party tools. The product is delivered with the embedded CSE HTML Validator LITE and PolyStyle Formatter. PhpED IDE includes a number of pre-configured tools like PHP documentor, HTML Tidy and a CVS client. • Support for international character sets, including UTF-8. True Unicode editing is now available. PhpED IDE can be used to create web sites in different encodings and natural languages. For all the latest info, visit NuSphere.com.
September 2005
●
PHP Architect
●
www.phparch.com
9
TIPS & TRICKS
Input Filtering, Part 3: Ensuring Input Received is Input Expected by Ben Ramsey
This year has seen an increased focus on PHP security, and this is good for the language, developers, and business community. One phrase that comes to mind when discussing secure coding practices is Chris Shiflett’s mantra of “filter input, escape output.” While we know what this means in a general sense, practical examples elude us. This month’s installment of Tips & Tricks concludes the series on filtering input, providing practical examples and helpful tips to filter input using regular expressions, test for the length of data, and ensure acceptable values.
P
art one of this series introduced the need to filter input and explained why all input, whether from a user or an RSS feed, should be considered tainted. I also introduced the whitelist approach as a best practice for filtering input. Part two further explained the whitelist approach, exploring the use of the ctype functions as excellent tools to implement a whitelistbased filter. Recall from parts one and two the HTML form used for discussion. I have included a modified version of
September 2005
●
PHP Architect
●
REQUIREMENTS PHP
n/a
CODE DIRECTORY
tips
this form in Listing 1. For the purposes of the present discussion, I have added the age, color , and username fields. Listing 2 shows the processing form as seen at the end of part two. Rounding out my three-part series on filtering input, this installment of Tips & Tricks includes discussion on using regular expressions to filter input, testing for the length of input, and ensuring the presence of acceptable values (e.g. from select, radio , or checkbox form fields, etc.).
www.phparch.com
Filtering with Regular Expressions In last month’s column, I discussed using PHP’s built-in character type (ctype ) functions to filter input. When application design allows, the ctype functions provide a fast and easy-to-use interface to implement a whitelist approach to filtering input. However, application design doesn’t always allow this, and the ctype functions lack flexibility. For example, ctype_alpha() only checks for alphabetic characters, 10
TIPS & TRICKS
Input Filtering, Part 3
Listing 1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
Name:
Street:
City:
State: <select name=”state”> Pick a state... Alabama Alaska Arizona
Postal Code:
Phone:
E-mail:
Age:
Color:
Blue
Red
Green
Yellow
Username:
Listing 2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
September 2005
●
PHP Architect
●
www.phparch.com
while ctype_digit() checks for only numeric characters. ctype_alnum() checks for both, but then it doesn’t allow for the presence of spaces, underscores, hyphens, or any other non-alphanumeric characters (nor do the previous two mentioned functions). On the other hand, ctype_print() is too open, allowing all printable characters, and this isn’t always a desired approach. When you know exactly what characters you want to allow, it’s best to restrict input to those characters—and only those characters. So, ctype_alnum() is good for usernames, and ctype_digit() is good for five-digit U.S. zip codes, but ctype_print() isn’t necessarily good for a first and last name, an email address, or a phone number. Good application design defines what characters these fields should accept; good filtering accepts only these characters. Enter PHP’s Perl-Compatible Regular Expression (PCRE) functions. These functions make up for their slowness—as compared to the ctype functions—with increased flexibility and power. Regular expressions can be used to match just about anything and can perform some amazing tasks. Take, for example, the name field in Listing 1. In Listing 2, I define it as a “string” type and then the filter() function filters it using ctype_print(). The decision to use ctype_print() over ctype_alpha() should be clear: I wanted to allow users to enter a space between their first and last names. However, now users can enter all sorts of random characters, characters that should not be acceptable for a name, so I turn to a regular expression to match a name. First, I come up with the following to replace the ctype_print() function: $clean[$key] = (preg_match(‘/^[A-Z ]*$/i’, $value)) ? $value : ‘’;
This works well for names such as “Ben Ramsey,” but suppose I want
11
TIPS & TRICKS
Input Filtering, Part 3
Listing 4 (cont’d)
Listing 3 1 2 3 4 5 6 7 8 9 10
Listing 4 1
deem acceptable. A better, “general purpose” regular expression for matching strings is: /^[-A-Z0-9\.\’”_ ]*$/i
I won’t go into the particular details of how regular expressions work. There are books and Web sites for
www.phparch.com
that, but I will share a few of my preferred regular expressions for filtering standard types of information, such as e-mail addresses, phone numbers, and postal codes. Looking back at Listing 2, I defined the postal code with the “int” type, which works well in certain circumstances when only the five-digit U.S. zip code is accept12
TIPS & TRICKS
Input Filtering, Part 3
able, but what if I want to accept a zip+4 postal code? These are typically written as “12345-1234,” and will cause ctype_digit() to return FALSE, because of the hyphen. Since the “int” type is useful in other situations (e.g. the age field), I won’t rewrite its definition. Instead, I’ll create a new type for “postal,” and create a regular expression to accept either a five-digit zip code or a zip+4 code (with or without the hyphen). /^(\d{5})[\-]?(\d{4})?$/
Likewise, the e-mail and phone number fields in Listing 2 are of the “string” type, but I know that there are acceptable patterns I want to match for both of these. Plus, my existing “string” regular expression doesn’t allow the “@@” symbol, or parentheses. Thus, I create an “email” type and define its regular expression as: /^[^@\s]+@([-a-z0-9]+\.)+[az]{2,}$/i
I also create a “phone” type, giving it the following expression: /^[\(]?(\d{3})[\)]?[\s]?[\]?(\d{3})[\s]? [\-]?(\d{4})[\s]?[x]?(\d*)$/
These two regular expressions will match most e-mail addresses or U.S. phone numbers. In fact, the expression used for phone numbers here can extract all the parts of a standard phone number to the matches parameter of preg_match(), if desired. It should be noted, however, that the e-mail address regular expression used above will not match some addresses considered compliant according to RFC 822 guidelines. Take the following input, for example: “JJohn Doe (home address) <
[email protected]>”. According to RFC 822 guidelines, this full string is acceptable, but the e-mail regular expression will reject it. Also, addresses that contain no TLD, such as jdoe@example, are valid RFC 822 addresses. If RFC 822 compliance is neces-
September 2005
●
PHP Architect
●
sary, then Listing 3 provides an alternative e-mail address filtering method using the PEAR::Mail package. This can also be accomplished using imap_rfc822_parse_adrlist() if PHP is compiled —with-imap. If portability is a concern, however, I suggest using the PEAR::Mail package.
rates all that I have discussed thus far. Notice how I have expanded $post_whitelist to include more information about each form field. Now, I associate an array with each field that defines the type of input to check against, in addition to several other details. One of those details is maxlength, which I check in the filter() function with:
Testing Input Length In part one of this series, I mentioned that, while the maxlength attribute of the HTML input tag controls how much data a user may enter when properly using a form located on the host site, it does not restrict the amount of data that a user may post when using a form located on another Web site, or when posting by some other means (see part one for more information). Likewise, client-side validation with JavaScript may provide good measure for practicing “defense in depth,” as well as a potentially better user experience, but it will not restrict the actual data that can be sent to the form processing script from somewhere else (e.g. another form on another Web site). Thus, it is necessary to perform all input filtering, or validation, on the server side, in addition to any client-side validation. Regardless of whether you filter input at the client, you must always filter input at the server. I have seen many sites that provide a maxlength attribute in their input tags but fail to test the length of the field from the server side. This leaves the processing script open to receive all lengths of data, which can lead to database constraint violation errors and, potentially, more dangerous issues. Checking the length of input, however, is simple, and, coupled with the maxlength attribute, it is easy to determine that a user is abusing the form if input received is longer than the expected length. Listing 4 is a finalized version of the filter() function that incorpo-
if (isset($whitelist[$key][‘maxlengt h’]) && (strlen($value) > $whitelist[$key][‘maxlength’])) { continue; }
www.phparch.com
Here, I use the continue statement to skip to the next item in the [foreach] loop, essentially excluding this value from the $clean array if it contains more data than expected. Since I have maxlength defined for these fields in my form, I am confident that no user using my form is able to enter more data than expected. If the input contains values that are longer than their respective maxlength, then I can assume that the user is abusing my form in some way, and I can safely exclude the input from the $clean array. Ensuring Acceptable Values In much the same way that maxlength cannot be relied upon to stop would-be attackers from sending unlimited amounts of data to form processing scripts, the values displayed in HTML select, radio button, and checkbox lists are not the only values that can be posted. Thus, it is necessary to filter the values of these fields and ensure that the input received is input expected. Again, this is not a hard practice to implement, but it does require more code. Take another look at Listing 4. In $post_whitelist, I’ve also added the “option” type, and for each item specified as type “option,” I have listed the expected options in the “options” array. For flexibility, I’ve also added the “multiselect”
13
TIPS & TRICKS
Input Filtering, Part 3
parameter that is defined on fields in which more than one item may be selected (i.e. checkboxes or menu lists). In the filter() function, under the “option” case of the switch statement, I check whether the input received is an array. If it is, then I further check to ensure that I’m allowing the user to select more than one item. If not, then the input received shouldn’t be an array, and I discard the data and move on. If it is a multi-select field, then I check to ensure that every item in the array matches those defined in the “options” parameter for the field. If it’s not an array, then I simply check to ensure that it matches one of the “options.” If it does, then I keep it; if not, then it is discarded. If a value is not acceptable—that is, it doesn’t conform to expectations—then I don’t keep it. It doesn’t get added to the $clean array. Notice how all values in Listing 4 are now set to NULL if they don’t
conform to expectations. Then, I check whether the value is null. If it is, I don’t save it to $clean. In part two of this series, recall that I did save it to the $clean array, with an empty value. I no longer do that, and, instead choose to completely discard the reference to the field. Now, the worst thing that can happen when working with user input is that a field doesn’t exist—but that’s easy to check and report. Moving Right Along Over the past three issues, I have given an in-depth look at input filtering in PHP. This discussion has covered such topics as “why to filter”, “using ctype functions and
regular expressions”, and “validating the length and acceptable values of received input.” I have discussed this all the while promoting a whitelist approach to ensure that input received is input expected. For future installments of Tips & Tricks, I would like to know what tips and tricks you are using. Please send your tip and/or trick to
[email protected], and, if I use it, you’ll receive a free digital (PDF) subscription to php|architect. Until next time, happy coding!
About the Author
?>
Ben Ramsey is a Technology Manager for Hands On Network in Atlanta, Georgia. He is an author, Principal member of the PHP Security Consortium, and Zend Certified Engineer. Ben lives just north of Atlanta with his wife Liz and dog Ashley. You may contact him at
[email protected] or read his blog at http://benramsey.com/.
To Discuss this article: http://forums.phparch.com/252
Award-winning IDE for dynamic languages, providing a powerful workspace for editing, debugging and testing your programs. Features advanced support for Perl, PHP, Python, Tcl and XSLT, on Linux, Solaris and Windows.
Download your free evalutation at www.ActiveState.com/Komodo30
FEATURE
Roll Your Own Database Abstraction Module by Jason Lustig
You may already use database abstraction in your applications, perhaps through one of the available database abstraction layers, such as PEAR::DB, or PDO (see the PDO article in this issue), but what about various idiosyncrasies in the actual SQL? Perhaps you’ve never even considered this problem. This article will help you the data abstraction beast.
H
ow does Adobe keep Photoshop working on both Windows and Mac OS, or Microsoft keep Office portable? Often, people take the route of maintaining separate codebases for different platforms. Mega-corporations have the resources to pull it off, but a smaller firm or even a lone coder probably couldn’t do it particularly efficiently. It’s one of the reasons why the Mozilla project decided to go with XUL as their frontend instead of maintaining different sets of code for Windows, Mac OS, Linux, and whatever else happened to come around. Prior to XUL, if the Netscape developers had to make a change, they had to update every codebase individually, and it was a major hassle. Web applications give us a little more freedom. HTML is fantastically portable—as long as there is a decent web browser for your desktop platform of choice, you will be able to access and work with your web applications. It has been argued that Microsoft has neglected Internet Explorer for exactly this reason: innovating too
September 2005
●
PHP Architect
●
www.phparch.com
REQUIREMENTS PHP
4
CODE DIRECTORY
abstraction
much in the browser space would kill the desktop, which is their big cash cow. Web applications are even more portable, on the server side, because most of the languages—be it PHP, Perl, or even some ASP, through emulators such as Chili ASP—can run on almost any web server in any operating system (within reason). The bottleneck to ultimate portability turns out to be the data itself. If you can abstract your data, then you will never be tied down again! This is, in a way, the “holy grail” of web application development: how can you make the database code portable but at the same time readable and hand-tuned for every database that you are writing for? How can I take advantage of lowlevel locking in Oracle when my MySQL code doesn’t
16
FEATURE
Roll Your Own Database Abstraction Module
even have transactions? How can I abstract my data to an extent that it can be used by all kinds of databases? It’s possible; I’ve done it. I was able to port my 200,000-line web application from MySQL to PostgreSQL in about two hours on a lazy Sunday afternoon. What Is Data Abstraction, and Why Bother? Data abstraction is when your application does not have to worry about where its data comes from. In the world of web applications, because most people use databases to handle their data, usually this translates into database portability—the ability for your application to interact with all different kinds of databases.
Server to mirror all of the data in our point of sale system, so that we can mess around and not have to worry about corrupting our actual, production data. This set up generally works pretty well. One day, we were having some trouble with the server, and my boss, who is pretty smart, and has some technical background, said “Would it help if we switched to Oracle?” The answer to this is, really, “I have no idea if it would help if we switched to Oracle.” We don’t need any of its fancy table locking features or anything like that, and SQL Server has been pretty good to us so far. It would be a lot of work to import our databases. The reason that I said “no, Oracle would not help us much at all,” is because we have many scripts, programs,
“The bottleneck to ultimate portability is the data itself. If you can abstract your data, then you will never be tied down again!”
Is portability worth it? In a perfect world, our computers would work properly most of the time, and we wouldn’t have any reason to switch operating systems, web browsers, etc. Why would we want to keep our code portable? Are there such big advantages that make the hassle worth the pain and suffering? (Because it is extra work to keep code portable, since you need to test across multiple systems.) It depends on your goals. There are definite advantages to portable code, such as opening up the market for your application to a larger group of people, avoiding lock-in, and more, but there are also disadvantages. Grow your market. If you are selling or otherwise making a computer program that other people will use, whether it’s a web-based application or not, it would be great to be able to offer it to more people. That’s the reason why the big guns (like Adobe) keep their software running on both Mac and Windows platforms. If they picked only one system to support, it would really cut costs, but would also alienate large group of potential customers. The more databases your web application supports, the larger the number of people who might be interested in purchasing or downloading it. Portability keeps your code more readable and more maintainable. If you use a modular approach to data abstraction, as I do, or even if you use an abstracted set of functions like query() instead of mysql_query(), then your code will be easier to read and maintain down the line. This is something nobody will argue against! Avoid lock-in. In my “real” job, I work in retail doing market research. Our current setup uses Microsoft SQL
September 2005
●
PHP Architect
●
www.phparch.com
nightly jobs, and other little bits of code written for SQL Server and fine-tuned to cater to its nuances and bugs. To port all of this code would take weeks and would not save us nearly enough time, in the long run. We’ve been locked in. Now, there is nothing particularly wrong with this, because we are doing everything internally and really there is no reason why we would want to switch to another database. But if we had to, we would really be in a bind. Unfortunately, it is much more difficult to keep code like this portable than it is to keep web applications portable. “Too Portable” or “Too Abstract?” It depends on what you are trying to accomplish. Just like many other processes that improve performance, grow your market, and make things easier to do, the concepts of data abstraction and portability function under the law of diminishing returns. What this means in English terms (as opposed to the economic mumbojumbo that it really consists of) is that as you make your code more and more portable, the benefit that you get out of it tends to decrease over time. So, when you first abstract your database, swapping mysql_query() for PEAR::DB, or another similar abstraction layer like ADODB, the relative increase in productivity will be greater than when you then go and abstract your queries, or do something crazy like begin to use an XML-based definition of your database structure. The key is to find a balance. You need to determine
17
FEATURE
Roll Your Own Database Abstraction Module
the point at which you are kidding yourself—where additional abstraction will cease to help you out. When you’ve reached this critical juncture, you should stop fussing around and get to programming your real application. This isn’t to say that abstracting your data isn’t worth it. But depending on the application you are writing and the job it is supposed to do, sometimes abstraction isn’t worth the time that you would spend to maintain it. A simple formula might be: the time spent maintaining data abstraction, divided by (the time it takes to
the language is standardized to a certain extent; you can assume that basic SELECT, UPDATE, and DELETE statements, JOINs, etc. will work on most modern databases. The tricky part with writing code like this is that you need to test it on all supported databases. When you make a change to the SQL, it might break some databases and not others. It increases the amount of QA work that needs to be done, while minimizing the amount of actual code you have to write. One important thing to remember is: databases are already a form of abstraction. They abstract away the
“Portability keeps your code more readable and more maintainable.”
write the application in the first place multiplied by the amount of time you plan to spend maintaining the application). If the result of this formula is greater than one, it probably isn’t worth it to abstract the data any more than you have to, in order to get it working properly without killing yourself with PHP’s arcane function names. Otherwise, it makes sense to abstract the data to your heart’s delight. Luckily for us, making a good abstraction layer is easy enough, and the learning curve is such that you can get used to it quickly enough, that the time to maintain data abstraction is usually low enough to guarantee that most of the time, it really is worth it. Let’s Get to Business, Shall We? We’ll begin with some simple pseudo-code to connect to a database, pull some data, and then display it. We are going to eventually abstract away different portions of the database code, in varying amounts, to try to find the “sweet spot” where we’ve balanced portability with the time we’ll spend on further abstraction. See Listing 1. Easy enough, right? We are already using some sort of basic database abstraction. We don’t call mysql_query() or postgresql_pconnect() anywhere in here; we have abstracted away the PHP functions so that we can rewrite the class to connect to an additional database. In fact, you might notice that the function names are similar to the ever-popular PEAR::DB abstraction class. It’s my personal favorite, because it is simple and takes care of most of the hard work for you, and at the same time it does not force you to abstract your database calls any further than you want. Additionally, the SQL code itself is pretty portable—
September 2005
●
PHP Architect
●
www.phparch.com
idea of data sitting on the disk in zeroes and ones, and think about it as tables and rows. SQL stands for Structured Query Language, and the theory is that it should be standard across all the different database engines. So, if you wrote your code with standard SQL, it should be portable… right? The problems that arise are often related to databasespecific extensions to the SQL standard. “Why use extensions?” you might ask. “Just stick to the standard—databases should be standards-compliant, just
Listing 1 1 2 3 4 5 6 7 8 9 10
Listing 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
18
FEATURE
Roll Your Own Database Abstraction Module
like web browsers!” Reality is that databases just aren’t always so standards-compliant. MySQL (before version 5) didn’t support stored procedures, and has a number of different table types, many of which handle locking and transactions differently. Oracle and Microsoft SQL each have a hundred handy little features that they have added to the standard which, in theory, make it easier to write applications. These features often serve as convenience functions, and allow you to do things like grab only one row, quickly. Why not take advantage of these extra features? If you don’t, you are just hurting your application by making it slower. But, if you have the SQL itself hardcoded into your main code, there is no way to really do this, right? Wrong. If you were so inclined, you could dynamically generate the SQL query, based on the database platform you are using. Say, for example, that you want to select the top ten rows from a table, and want to support both MySQL and Microsoft SQL Server. These two databases use different syntaxes to limit the number of rows returned from a query. SQL Server uses “ttop xx” and MySQL uses “llimit xx”. However, the code in Listing 1 could be adapted to support both databases, as in Listing 2. Easy enough, right? In theory, yes, but it makes your code impossibly hard to maintain, especially if, one day, you decide that you also want to support Oracle, PostgreSQL, Firebird, and maybe also DBase or SQLite. Additionally, it is less secure because it opens the door to making some big mistakes, since you are always gen-
erating the SQL statement on-the-fly. What if you mess up and put something inside the “$$sql .=” portion that shouldn’t be there? This opens the $sql variable up to a possible injection attack. It is a hacker’s paradise. Roll Your Own Language Let’s say you just want to have one set of database code to rule them all. You could go the route of abstracting the idea of your query, and then write a class that will generate the SQL as necessary. You could add the ability to set optimization flags, if the database can handle it. Depending on the database, your SQL generator will either pay attention to or pretend these flags didn’t exist. Let’s look at the same code again but with a made-up SQL generator (Listing 3). In this latest attempt to abstract our database query, we have gone to great lengths to tell our code what we are trying to do. Essentially, the db_query::generate() function can figure out which database we want to talk to, and create an optimized query at will. You don’t even need to use a function-based abstraction; you can create XML files that describe your queries, or even your entire database structure, making it human-readable, as well. But is it the best way? Personally, I don’t think so. You end up just writing your own query language that needs to be debugged and audited for security. You’d have to maintain another complex abstraction layer in your application, when you could instead be writing
Figure 2
Figure 1
A Unified Binary contains executable code for both the x86 and PowerPC (PPC) architectures in one file
An Ideal Web Application Figure 3
The directory structure of our application makes it easy to make new data abstrac-tion modules and to differentiate between them.
September 2005
●
PHP Architect
●
www.phparch.com
19
FEATURE
Roll Your Own Database Abstraction Module
another simple layer. All too often, people over-abstract their applications and focus too much on the framework and not on making features that make their application cool and fun to use. Unified Binaries, Unified Abstraction Over the past few years, while working on various applications, I have developed a method which, in my opinion, is the best that I have seen. It’s a system that allows you to create new modules—or port your code to new databases—quickly and easily. In fact, this method makes it so that your “core” application never actually touches the database or whatever sort of data store you’re using. This opens up all sorts of interesting possibilities, because your application doesn’t care which database stores the data. It really doesn’t even need to be a database. You could write a module that stores your data in flat files, or even shared memory, if you wanted to. We’ll cover that, later. Most people will agree with the idea that modular applications are a good thing. This “ideal complicated program” is made up of modules that interact with each other, through interfaces, abstracting away the ugliness of any code that may reside underneath. Other portions of your program can assume (within reason) that this abstraction layer simply works and you will never have to think so much about what’s actually happening. You will only need to work with the data that is returned from the modules. Using a simple, standard way of returning errors from the database modules, failures can be easily handled, as well. For those who don’t understand all that architecture mumbo-jumbo, let’s draw a picture of this “ideal” web application (Figure 1). The idea, here, is that each level of the application takes care of one aspect of displaying a page, whether it is generating the HTML code that is sent to the browser (templates), the “business logic,” sanitizing users’ input data, or anything else that a typical web application must do. This makes “n-tiered applications”, where n represents the number of tiers (also known as levels, modules, or by many other names). The most popular and well-known of these n-tiered models in the web application space is the three-tiered application, also known as Model-View-Controller. In the Model-View-Controller architecture, you have three levels: a database (“model”), the business logic (“controller”) and HTML generator (“view”). There are many benefits to this model, especially in terms of scalability. You can put each of these three tiers on different groups of servers, and if you need to be able to support more users, just throw more hardware at your application. Having multiple-tiered applications is great for other reasons as well, including cleaner code, and better documentation. You can also pull out modules and replace
September 2005
●
PHP Architect
●
www.phparch.com
Listing 3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Listing 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Listing 5 1 2 3 4 5 6 7 8 9 10
Listing 6 1 2 3 4 5 6 7 8 9 10
sql__select_top10_from_mytable() database connectivity class $db; ‘select * from mytable limit 10;’; $db->query($sql);
Listing 7 1 2 3 4 5 6 7 8 9 10 11 12
20
Roll Your Own Database Abstraction Module
them with others that have the same API, but work in a totally different manner, underneath. This is where it gets interesting with regard to our data abstraction problem. Another other great advantage of web applications is that, for the most part, especially if you use a language like PHP, they are dynamically compiled and run. This means that you can interchange files at will, and users will not be able to tell the difference. We’ll take advantage of this, to create multiple database modules that work along a set interface to our business logic. In this way, to create a new module—in other words, support a new database system—all we need to do is port one database module’s code to the new database, and voila! Your application has been ported to a new database. Usually, when people talk about database modules, it is for the most part constrained to database connectivity, as we looked at before. Connectivity defines how your application talks to the database and sends queries and other messages back and forth. We can use one if we want to with this system, but ultimately, because of our modular system, it does not matter where we are getting the data nor where we are storing it, so long as it conforms to the set interface that our “business logic” knows how to deal with. This modular tier that I propose won’t live on a different server (though you could put the files on one), because it is actually a part of the “controller” level of the application. It is surprisingly similar to Apple’s “Unified Binary” approach to compiling programs for both the PowerPC and x86 CPUs, which is why I like to refer to it as “Unified Abstraction.” What is a “Unified Binary,” how does it work, and what does this have to do with data abstraction? Well, Apple has a peculiar situation coming up where it will be supporting two CPU families: IBM’s PowerPC, which is what Macintosh computers have used for the past ten years or so, and Intel’s Pentium (x86) family of processors. This presents a major problem for software developers. What are you going to do about developing for both processors, since a binary compiled for PowerPC won’t run on x86, and vice-versa? It’s a very similar problem to our issue with databases. The solution that Apple came up with is this: within the “application” that you create is really two binary programs. One is compiled for the PowerPC processor, and the other for x86. When you open up a Unified Binary, Mac OS will just use whichever binary is compatible with your computer, and it can use resources (internationalization files, images, etc.) normally, because they are just normal files. We will use a similar method. When you create the PHP script for your web application, you will write it as a core file that doesn’t really care about which database you’ve chose; this is similar to the resource files. It figures out which database we are working with, and then September 2005
●
PHP Architect
●
www.phparch.com
FEATURE calls the appropriate database module, which is analogous to different binaries for PowerPC and x86. The key is that your application somehow needs to know which database it is using. Somewhere, you are storing the database connection credentials, such as the username, password, hostname, and so on. In this same place, you can keep information about whether you are connecting to MySQL, Oracle, Microsoft SQL Server, or even a flat-file database. We can add an extra line, $dbtype = ‘mysql’, to our examples. The main scripts that live in your web-root, which is what people see when they come to your site, won’t contain any actual database calls. Rather, they call functions that return database records. Alternately you can use an object-oriented approach, though I prefer simple functions because they lead to less code, which is, in turn, less complicated. In Listing 4, the sql__select_top10_from_mytable() function ideally return a PEAR::DB_Result object. We use the DB::isError() function to check to make sure that our query worked properly. You may have noticed that Listing 4 won’t run because it’s missing the declaration of the sql__select_top10_from_mytable() function. This is because the listing contains only the core script, which hasn’t yet called the database module. Let’s create a script called dbtest.php, and place it in our application’s root directory. We could create a subdirectory called sql, and within that, another directory, mysql, pgsql or whatever we want. This nested directory would contain our database module. In that way, we can create new modules simply by creating another directory beneath sql, such as mssql or oracle. How does the PHP file know where to find the SQL file associated with it? Listing 5 shows a function that performs this task. Within our main script, we can just add the line require_sql(‘dbtest.php’); and our file will be included. Within sql/$dbtype/dbtest.php is the function shown in Listing 6. Of course, you could name the function anything you like, but I usually choose to preface them with sql__ (and then, usually with something dealing with the name and location of the associated core script, in a larger application), because this way, functions won’t have the same name, thus avoiding naming conflict. You could also pass it variables, as shown in Listing 7. In this way, you could have a similar function that uses the alternate method of limiting rows, within the mssql module. Your application would be none the wiser—it would just proceed as normal, and wouldn’t care at all if you used limit or top within the query. You can optimize each query for each specific database, as much as you like, and you’ll not have to worry about the fact that all those obscure keywords might fail on another database system.
21
FEATURE
Roll Your Own Database Abstraction Module
To port your application to a new database, all you’ll need to do is take the database module whose SQL syntax is closest to the one you are porting to, duplicate its directory within sql/, and rename it appropriately (to e.g. oracle or dbase, etc.). Then, just go in and change the SQL calls so that it takes advantage of the new database’s features, and voila! You now support a new database type! The reason why you would start by copying the module for the database whose syntax is most similar to your new database is to require the fewest possible changes to the SQL within the module. Maintanence “Alright,” you might be saying, “this sounds interesting, but also it seems like a lot of work to maintain!” It really isn’t that much work, once you’re used to it. When you want to change a database query, you just need to change the SQL in each of the database modules. It’s also easy to add new functions, because, if you first write only a simple function that doesn’t use advanced and non-portable features of your favorite database, you can just copy the function over to your other modules and then go and make each one take advantage of your table hints or other bells and whis-
tles. Of course, if you keep your database modules welldocumented, maintenance is easier, as well. Conclusion Data abstraction can be done in many ways. The method that I have suggested is one that I personally prefer because of the ease of porting applications to new databases and data storage methods. It isn’t for everyone or for every project—just like some quickand-dirty applications don’t necessarily separate content from logic using templates, sometimes abstraction isn’t worth it. Database abstraction, at the SQL level is one of those things that doesn’t usually hurt too much, and helps out in the long run.
About the Author
?>
Jason Lustig is a student at Brandeis University in Boston. He is a freelance programmer who dabbles in database and application design, and works part-time doing market research and data mining.
To Discuss this article: http://forums.phparch.com/248
Available Right At Your Desk All our classes take place entirely through the Internet and feature a real, live instructor that interacts with each student through voice or real-time messaging.
What You Get Your Own Web Sandbox Our No-hassle Refund Policy Smaller Classes = Better Learning
Curriculum The training program closely follows the certification guide— as it was built by some of its very same authors.
Sign-up and Save! For a limited time, you can get over $300 US in savings just by signing up for our training program! New classes start every three weeks!
http://www.phparch.com/cert
September 2005
●
PHP Architect
●
www.phparch.com
22
FEA TURE
An Introduction to
PDO by Ilia Alshanetsky
A common complaint of the anti-PHP “expert” is the lack of a bundled, uniform database access component. With the advent of an improved object model, in PHP 5.0, a few of PHP’s core developers decided that the time has come to fill this hole with PHP Data Objects (PDO). The pack-
age, itself, has been in PECL for quite a while, now, but with the upcoming release PHP 5.1, PDO will be bundled in the main PHP distribution. What does it do? How does it work? One of PDO’s main developers explains.
N
While this approach has proven to be quite affective over the years, it does pose one particular problem: the PHP APIs for talking with most databases are relatively similar, but are far from identical. This problem is most apparent in the functions defined by the various database extensions. Each has its own, distinct, set of functions. For example, the MySQL extension uses mysql_fetch_row() to retrieve a record as an array of elements, while PostgreSQL makes use of pg_fetch_row(). Aside from the differences in the names, the parameter order of the functions is also eclectic. Using MySQL and PostgreSQL as examples, the former’s query execution function does not require a database connection resource—and if one is provided, it takes the last position in the function call’s parameter list. In PostgreSQL, and several other extensions, a database resource is required, and must be supplied as the first parameter to the function. Document the differences between the various extensions would proba-
early everyone who has ever employed PHP has used it to talk to a database system. In most cases, a database provides a highly flexible and capable information storage and retrieval engine, ideal for data gathering and analysis. It is really no wonder that databases use is so prevalent in the developer community. As with most popular tools, there are often multiple approaches to the same problem, and database systems are no different from the norm. There are literally dozens of different database systems all competing for your attention as the best way of dealing with information. PHP—the language of choice for millions of developers—unsurprisingly supports the majority of these database engines, to ensure that no one is left out or feels neglected. In most instances, the development of a database interface in PHP is not the result of a master plan or even a consequence of a well-planned specification, designed to provide the ideal method of database communication. More often than not, it is the result of a situation where a developer needed to have PHP connect to a previously unfamiliar database. By taking some existing code, possibly from other database extensions, and adjusting it to work for their particular database, the developer creates an initial interface. Usually, other users and developers then come up with tweaks, additions and refinements to the initial code base that eventually evolves into a full database extension. September 2005
●
PHP Architect
●
www.phparch.com
REQUIREMENTS PHP
5.0+
OS
N/A
Other Software
PDO and an appropriate driver: http://pecl.php.net/pdo
Code Directory
n/a
23
FEATURE
An Introduction to PDO
bly require an entire book, and is far beyond the scope of this article. The API difference is something that is of little concern the developers who only communicate with a particular database; it does, however, present a serious problem to those who need to support multiple database back-ends. This has lead to the creation of numerous database abstraction libraries. These range from simple ones that merely choose the right native function for the job, and possibly juggle the arguments, to complex and ultimately slow beasts that not only abstract the interface, but also try to handle various incompatibles between the database systems, themselves. This has been somewhat of a pet peeve for the PHP core development community. This is why we decided to address the issue—during LinuxTag 2003—with the advent of PHP Data Object (PDO). PDO was designed to use the latest PHP 5 object orientation support to provide a common API for all database systems with which PHP can communicate. By creating a common database communication interface, the need for the majority of database wrappers is eliminated. Because it was written in C, rather than PHP, the interface is very fast, and has very minimal—if any— overhead to the native interface. Furthermore, PDO aimed to identify common operations that are performed on a database, and provide easy and convenient means of applying (or emulating if necessary) them, for all supported databases. These abilities include: • execution of INSERT/UUPDATE/DDELETE queries • retrieval of data from a database in various forms: • as an array • as an object (new of pre-existing) • into bound variables • as a string • retrieval of all rows as a multi-dimensional array • prepared statement querying • the use of transactions • auto-commit support • the ability to normalize the case of table columns Thus, the “only” thing the code author needs to worry about is the differences in the databases themselves, which is simple enough as long as you use standard SQL. Current State of Affairs At this time, PDO has reached the majority of the initially-set goals and offers nearly all of the initiallyplanned features. It also includes support for all major databases with
September 2005
●
PHP Architect
●
www.phparch.com
which PHP can communicate: • MySQL 3 and 4 (ppdo_mysql) • PostgreSQL (ppdo_pgsql) • SQLite 2 and 3 (ppdo_sqlite – in fact, PDO is the only way to connect PHP to SQLite 3) • Oracle (ppdo_oci) • Firebird (ppdo_firebird) • MSSQL and FreeTSD (ppdo_dblib) • ODBC (ppdo_odbc) All of the drivers (with the possible exception of the Firebird driver) are quite stable and are regularly tested for both bugs and functionality. At the present time, some are already being used on production systems. Nonetheless, PDO and its drivers are a relatively new addition to PHP, and as such, may contain some yet-tobe-discovered bugs, so consider yourself warned. Installing PDO How do you get PDO? In PHP 5.1 (which should be out shortly), the PDO core extension and its SQLite driver are enabled by default. Other drivers are part of the standard distribution; however, they need to be explicitly enabled via a configuration switch. These usually are in the —with-pdo[database_type]=[interface_lib_path] format. For example to enable MySQL support you would use the –-with-pdo-mysql=/usr/local/mysql, assuming that the MySQL client library can be found in /usr/local/mysql. For PHP 5.0.X users, the situation is a bit different. Because PDO is not part of the standard distribution, it must instead be downloaded and installed from the PECL repository, or downloaded in binary form (for Win32 users), from http://snaps.php.net/. For installation from PECL, you simply need to execute the following commands: pear install pdo pear install pdo_[driver] #(example: pear install pdo_sqlite)
Upon execution, these commands will download the latest stable PDO release, and then automatically compile it. The next step involves loading the compiled PDO modules into PHP via php.ini: //*NIX users extension=pdo.so extension=pdo_sqlite.so // Win32 users extension=php_pdo.so extension=php_pdo_sqlite.so
In PHP 5.0.x, there is no automatic handling of module dependencies; therefore, it is absolutely imperative that the PDO extension, itself, be loaded prior to any of its drivers. Failure to follow the correct loading sequence will usually result in a prompt crash, due to the driver
25
FEATURE
An Introduction to PDO
attempting to access information that is not yet available. PHP 4 users are, unfortunately, out of luck. PDO relies heavily on OO features only found in PHP 5 and higher, and simply does not work on previous releases. Starting to use PDO The first step in using PDO is not too dissimilar from using any other database interface. This procedure requires the creation of a database connection handle, which in the case of PDO, involves instantiation of a PDO object. The constructor of the object takes a number of parameters, but the only required argument is the DSN. The DSN, in most cases, defines the hostname
“One trick with PDO is the ability to iterate through PDOStatments, via the foreach construct.” and the database to talk with. For some databases like PostgreSQL and Firebird, it can also be used to specify the login and password—however for most databases, this information is supplied via the 2nd and 3rd arguments to the method, respectively. The constructor also takes an optional 4th argument that can be used to specify an array of attributes. These additional directives can only be set during the connection initiation phase, and adjust the entire connection for features like auto-commit, and set regular attributes like error reporting mode, etc… // MySQL connection new PDO(‘mysql:host=localhost;dbname=testdb’, $login, $passwd); // PostgreSQL new PDO(‘pgsql:host=localhost port=5432 dbname=testdb user=john password=mypass’); // SQLite new PDO(‘sqlite:/path/to/database_file’);
When it comes to the DSN parameter, it must always start with a database identifier, such as mysql: that allows PDO to determine which underlying driver to use. The remaining attributes indicate the actual connection parameters. In most cases—as demonstrated with the MySQL driver—the connection tokens are separated with a semicolon. One notable exception is
September 2005
●
PHP Architect
●
www.phparch.com
PostgreSQL, where the database client supports its own DSN style, natively, so PDO supports the native format to make things easier. SQLite is another exception to the rule: the database is just a file, so the only token (aside from the driver identifier) is either the path to the database or the special “::memory:” string for memorybased databases. The connection process is the really the one place in PDO where differences between databases are exposed—the rest of the code is standard. As with most object oriented extensions, failure during object construction—which translates to connection failure in this case—will cause PDO to throw an exception of type PDOException. A thrown exception is something that you definitely want to catch. Uncaught exceptions, in PHP, result in PHP’s native engine raising a fatal error, which terminates the currently-running script. try { $db = new PDO(…); } catch (PDOException $e) { echo $e->getMessage(); }
In most cases, the message component of the exception should provide sufficient information to indicate why the connection to the database could not be established. As with all exceptions, additional debug methods are available that can be used to gather the location of the code that has triggered the exception via the getFile() and getLine() methods. It is even possible to get the list of function and method calls that lead up to the offending code, via the getTrace() method. In some cases, it may be undesirable to include the database authentication DSN string directly inside the script. For those situations, PDO provides two alternatives to the default mode demonstrated in the previous examples. One approach is to use an INI setting that can store the entire DSN, and then be referenced via a special “name” token, which aliases the pdo.dsn.name configuration directive. The INI directive is without any scope restrictions, meaning that it can be set in php.ini, httpd.conf or .htaccess on Apache servers or even defined via ini_set() within the script itself. ini_set(“pdo.dsn.name”,”sqlite::memory:”); $db = new PDO(“name”);
As you might have guessed, the token is actually the last part of the INI setting name. So, you could easily do PDO(“ilia”), in which case the DSN will be fetched from the pdo.dsn.ilia INI directive. While the naming convention is somewhat amusing, this does have practical uses. By being able to use a custom name, each application can define its own connection string, without creating conflicts. For example, FUDforum can use pdo.dsn.fudforum, while phpMyGallery—which could be running on the same virtual host—would use
26
FEATURE
An Introduction to PDO
pdo.dsn.gallery, and so on. Another way to denote the DSN involves the use of the uri: prefix, followed by the path to a configuration file that contains the connection string. This method is a bit inefficient, since it requires an extra file access for every database connection attempt, but in some cases it may be worth it. $db = new PDO(“uri:/etc/app/config” . md5($_SERVER[‘DOCUMENT_ROOT’]));
For example, let’s say you want to install 50 instances of the same application, all using the same code base but different databases. Each application can be made to look for a separate DSN, which could be determined by the md5 hash of the document root where the application runs. In a scenario like this, the code does not need to be altered in anyway way and the custom configuration files can be easily generated by the installation process. Executing Queries Once the database connection is established, and the PDO object is available, a number of operations can now be performed via a variety of methods. These methods include exec(), which was designed for execution the of queries that perform an operation, but do not return a record set. Examples of this type of query are the UPDATE, INSERT, and DELETE operations. Upon successful execution of the query, the exec() method will return the number of rows that were affected by the operation. If no rows were affected, the value of 0 will be returned, and in the event that the query failed, due to an error, the function will return a boolean FALSE. $rows_affected = $db->exec(“INSERT INTO my_table (row1,row2) VALUES(1,2)”); if ($rows_affected === FALSE) { // query has failed $einfo = $db->errorInfo(); echo $db->errorCode().”: “.$einfo[2].”
\n”; }
Because PHP is a type-insensitive language, a casual comparison will cause both FALSE and 0 to evaluate to the same thing, due to internal type normalization. Subsequently, if in my code example I had used “iif (!$db->exec())”, or “$$db->exec() or die();”, the error condition would be triggered for a perfectly valid operation that didn’t actually fail. To avoid this situation, the “error check” is performed as a separate operation that compares the returned value to FALSE in a type-sensitive manner via ===. In the event of an error—such as a failed INSERT— there are two methods available for the purpose of retrieving information about the cause of the failure. First, we have the errorCode() method that returns an appropriate SQLSTATE code—a 5 byte long alphanu-
September 2005
●
PHP Architect
●
www.phparch.com
meric string, indicative of the error that has occurred. SQLSTATE codes are a cross-database standard for reporting errors, and by returning them, PDO provides a database-independent, and consistent way of identifying errors. If a bit more detail about the error is required, the errorInfo() method can employed to return database’s native error code and error message as the 2nd and 3rd array elements, respectively. This is generally useful in situations where queries have failed due to syntax problems and you want to see the part of the query that the database was incapable of parsing. PDO’s Approach Error Handling While we are on the topic of errors, let’s quickly examine how the PDO extension handles problematic situations. With the exception of a failed connection, which results in a the throwing of an exception, PDO keeps quiet regarding errors, and—unlike many other database extensions—does not emit warning or error messages. The failed operation simply returns FALSE and leaves it up to the developer to detect and handle the situation. This is, however, something that can be easily altered, by changing PDO’s error handling mode via the setAttribute() method. This method is the primary mechanism for changing PDO settings (although, settings can also be changed via the 4th parameter of PDO’s constructor). The attribute of interest in this case is PDO_ATTR_ERRMODE, which controls the error handling. The possible values are: • PDO_ERRMODE_SILENT: the default mode of operation, where errors and warnings are not raised • PDO_ERRMODE_WARNING: triggers warnings when an operation fails, and • PDO_ERRMODE_EXCEPTION: makes PDO throw exceptions on any failed query // an example of PDO::setAttribute $db->setAttribute(PDO_ATTR_ERRMODE, PDO_ERRMODE_WARNING);
Settable Attributes Settable attributes in PDO are divided into three sections. The first of which is “connection time only” attributes, whose values can only be specified when establishing the database connection. These include PDO_ATTR_TIMEOUT, which defines the maximum number of seconds PDO will wait for the database system to respond, and PDO_ATTR_PERSISTENT, which can be used to toggle the use of persistent connections. Unlike the native drivers, PDO does not require a separate driver—the connect operation handles this:
27
FEATURE
An Introduction to PDO
new PDO($dsn, $login, $pass, array(PDO_ATTR_TIMEOUT=>5,PDO_ATTR_PERSISTENT=>1));
The persistent connections implementation in PDO has an additional feature, which may seem a bit unusual, but is actually quite useful. This feature gives developers the ability to “name” the persistent connection, by specifying a string value as the persistent setting’s attribute. This functionality allows multiple, completely distinct persistent connections to exist within the scope of a single script: new PDO($dsn, $login, $pass, array(PDO_ATTR_PERSISTENT=>”con2”));
There are certain standard settings that are supported by all PDO drivers, such as PDO_ATTR_CASE, which defines the case of the column names, as well as PDO_ATTR_ERRMODE, which we’ve already covered. The former ensures that when it comes to retrieving data into an associated array, the keys—which are based on column names—are predictable. Different database systems have different rules pertaining to the handling of column names; this makes a normalization routine necessary for consistent database behavior. The final class of attributes is intended for setting options that are specific to a certain database. These, in most cases, are used to expose database-specific features via PDO, or overcome the limitations of a given native driver. One such attribute is the long-winded PDO_MYSQL_ATTR_USE_BUFFERED_QUERY, that enables the use of buffered queries within the MySQL driver. By default, the MySQL driver uses unbuffered queries, which are more memory efficient, but prevent you from working with multiple result sets at the same time. You can usually spot attributes in this class by checking to see if their name includes the name of the database. Since there is a way to set attributes, logic dictates that there would also be a way to determine their existing values. In PDO, this is made possible via the getAttribute() method, which takes the attribute constant and returns its current value. In addition to the settable attribute, this mechanism can also be used to retrieve the information about the database you are currently working with. For example, the PDO_ATTR_SERVER_INFO attribute can be used to retrieve the information about the server with which you are communicating, and the PDO_ATTR_SERVER_VERSION will return you the server’s version string—e.g. 4.1.11-max for MySQL. Another handy, non-settable attribute is PDO_ATTR_CONNECTION_STATUS, which enables retrieval of the connection status. This particular attribute becomes especially useful in instances where persistent connections are being used, as those may timeout due to extended periods of inactivity. It is possible to use this attribute to easily determine if the persistent connection acquired by PDO is usable or not.
September 2005
●
PHP Architect
●
www.phparch.com
Post-INSERT Record ID Retrieval Reversing a little back, let’s come back to our INSERT query example that we performed via exec(). Many database systems support an auto-incremented ID that is added to every inserted row. This ID allows for a quick and simple identification of the row, for all sorts of purposes. In MySQL, this is done by setting the column specification to INTEGER AUTO_INCREMENT PRIMARY KEY, in SQLite, it’s as simple as INTEGER PRIMARY KEY, and in PostgreSQL, a SERIAL column type that is attached to a sequence facilitates this need, and so on. It is a common behavior to retrieve this auto-created value so that the inserted record can be associated with another subset of data. PDO offers the lastInsertId() method, to facilitate the retrieval of this value. The method, when executed at the end of a successful insert, returns the row identifier. $db->exec(“CREATE TABLE my_table “ . “(id INTEGER PRIMARY KEY, a INT,b INT,c INT”); $db->exec(“INSERT INTO my_table (a,b,c) VALUES(1,2,3)”); $id = $db->lastInsertId();
There is a bit of a peculiarity with this functionality, when it comes to PostgreSQL. By default, the value returned is the OID id, which is an internal row counter assigned to each table record. This value is not the value of the current SERIAL column; however, it can be resolved via a query such as this: $oid = $db->lastInsertId(); $q = “SELECT id FROM tbl_name WHERE oid={$oid}”; $id = $db->query($q)->fetchColumn();
This is, however, rather inconvenient, and is somewhat slow, as the process requires execution of an additional query and retrieval of fetched results. To improve the situation, the authors of the PostgreSQL driver added an optional parameter to the lastInsertId() method which is the sequence name for the affected table. By specifying this parameter, you allow the underlying code to query the sequence, directly, and make the method return the desired ID right away. The sequence names (which are generally picked automatically by the database) are predictable, given the name of the table. For example, given the counter column name of id and the table name of foo, the generated sequence name would be foo_id_seq. Transactions When performing database modification operations, such as inserts and updates, it is often necessary to maintain consistency between operations. One common way to address this is by grouping queries into transactions, which maintain data integrity. Another benefit of transactions is the ability to undo, or as it is referred into in industry terms, rollback a transaction if an error occurs while processing the
28
FEATURE
An Introduction to PDO
queries found within. This would result in the reversal of the affected tables, to the state they were in, prior to transaction initiation. On the other hand, if all queries were performed successfully, then the changes can be committed to the database in a quick and consistent manner. To encourage transaction use, PDO provides three methods for working with them. while (1) { $db->beginTransaction(); // start transaction for ($i =0; $i < 10; $i++) { if ($db->exec(“INSERT INTO foo …”) === FALSE) { $db->rollBack(); // query failed, abort break 2; } } $db->commit(); break; }
The beginTransaction() method, as the name suggests, initializes a new transaction, inside which any number of queries can be executed. In the event that a query fails, the transaction can be aborted via the rollBack() call, or if no errors were detected, the queries can be committed to memory via the commit() method call. Data Retrieval Now, we get to the interesting part: data retrieval. Here, PDO shows, by far, the greatest amount of flexi-
In most cases, having both sets of keys is somewhat pointless and quite inefficient. So, the fetch() method allows you to specify the desired array keys to be used, via function’s parameter. This can be either PDO_FETCH_NUM, in which case only numeric keys (fastest) will be used, starting from key 0—the first column—and incrementing by one for each subsequently found column. Alternatively, if you want to use slightly slower, but more user-friendly associated keys, you can set the fetch mode to PDO_FETCH_ASSOC . Another possible fetch mode is PDO_FETCH_OBJ, in which case, the returned row will be represented in the form of an object—an instance of stdClass— where column names become the object properties. $rows = $db->query(“SELECT id FROM foo”); while ($row = $rows->fetch(PDO_FETCH_OBJ)) { var_dump($row);// object(stdClass)#1 (1) {[“id”]=> int(1)} }
In case these three fetch modes are not enough, PDO introduces yet another fetch mode, PDO_FETCH_LAZY. In many situations where results are fetched, only a portion of the data ends up being used. Ideally, the extra, unused data would be simply left unselected, but it may sometimes be needed for certain conditional operations. Lazy fetch allows you to retrieve the row as an object—akin to the PDO_FETCH_OBJ mode. This mode, however, only populates the properties with their respective values when they are being accessed. This
“When it comes to data retrieval, PDO shows the greatest amount of flexibility and capability.” bility and capability compared to any other database extension. But, before we get to data retrieval, let’s quickly examine the process by which we can execute queries that need to return a record set. For data retrieval queries that are to be executed only once throughout the script, there is a query() method. This method takes the query string as the first parameter, and if the query execution is successful, it returns a PDOStatement object that represents the fetched result set. The actual selected data can be extracted from this object, in a number of ways. One approach is to use the fetch() method, which by default will return each row as an array with both the numeric (column position) keys and string (column name) keys. $rows = $db->query(“SELECT id FROM foo”); while ($row = $rows->fetch()) { var_dump($row); // Ex: array(0 => 1, id=>1) }
September 2005
●
PHP Architect
●
www.phparch.com
means that if, while working with a result containing 10 columns, you’ve only used 5, PHP would only allocate memory for the 5 columns that were used, reducing the overhead involved in the data retrieval process. The object representation of a result set does not necessarily need to use the default stdClass. PDO provides a way to create an instance of any object, and populate its properties with the retrieved values. The column names will be used to reference object properties, as is the case with the default functionality. If a property is already defined, it will be assigned a value, and if no existing property with a matching name is found, it will be created, dynamically. This functionality is exposed via the little-known fetchObject() method that takes a class name as the first parameter and an optional array of arguments to pass to the class’ constructor. $stmt = $db->query(“SELECT * FROM user WHERE id=1”); $reg = $stmt->fetchObject(“user_data”);
29
FEATURE
An Introduction to PDO
Using retrieval mechanism, we’ve filled $reg with an instance of the user_data class, populated with data from columns found in the user table. Object-based data retrieval is not limited to the creation of new objects for each result set—an existing object can be populated with the retrieved data. This makes for a much more performance-friendly solution, since object creation can be a slow process. To accomplish this trick we need to call the setFetchMode(), method which is used to set the retrieval mode. This mechanism provides a bit more flexibility than passing the mode via fetch(), by allowing us to specify another mode-related value, which, in this case, is the class instance. $reg = new user_data; $stmt->setFetchMode(PDO_FETCH_INTO, $reg); while ($stmt->fetch()) { // $reg == user_data class filled with from-db data }
To further simplify and accelerate the data retrieval process, PDO offers the fetchAll() convenience method within PDOStatement objects. It works pretty much in the same was as fetch(), except instead of retrieving only a single record, fetchAll() will retrieve all records from the result cursor, in the form of an array. Each array element will be an array or an object, depending on the fetch mode that is specified via the method’s single parameter. $rows = $db->query(“SELECT id FROM foo LIMIT 1”)>fetchAll(); print_r($rows); /* array ( array(0 =>1, ‘id’=>1) ) */
As with fetch() , the default retrieval mode is PDO_FETCH_BOTH. The main goal of this function is to simplify the process to retrieving small results sets, where it would be faster to create an array of results and then iterate through the set, calling the fetch() method for every found record. In instances where the result comprises of just a single column, the process can be optimized even further. Rather than retrieving an array, the value can be fetched in the form of an immediately usable string, by specifying the PDO_FETCH_COLUMN fetch mode. When combined with fetchAll(), it provides an immediately usable array of values that can be accessed, directly. For example if I wanted to see the complete list of tables starting with prefix “ffoo_” (in MySQL), I could simply execute the following bit of code: $tables = $db->query(“SHOW TABLES LIKE ‘foo_%’” )->fetchAll(PDO_FETCH_COLUMN); print_r($tables); /* array( “foo_bar”, “foo_baz”,
September 2005
●
PHP Architect
●
www.phparch.com
… ) */
Even though PDO_FETCH_COLUMN is supported by the fetch() method, PDO offers a dedicated single column fetching method, fetchColumn(). In the default mode of operation, this method will fetch the first column from the result set and return the retrieved value in the form of a string. If the desired column is not first one, you can specify the numeric position of the column via an optional argument. Keep in mind that, as with most things in PHP, the column count begins at 0. $rows = $db->query(“SELECT id FROM tbl”); while ($id = $db->fetchColumn()) { … }
When the result set is comprised of a single row, the column value can be dereferenced, directly from the return PDOStatement object, in a way similar to the one I’ve used with fetchAll(), in the previous examples. This is something that is possible, due to the improved object support in PHP 5, which allows access to an object, directly as the return value of another operation. $qry = “SELECT id FROM users WHERE login=’”. $db->quote($_POST[‘login’]).”’ AND passwd=”’.md5($_POST[‘pwd’]).”’”; if ($db->query($qry)->fetchColumn()) { show_login_prompt(); exit; } else { create_user_session(); }
In the above example, this functionality simplifies the process of validating the user authentication information, and determining whether or not the user should be logged in to the system or prompted once again, for authentication information. Special Character Handling The previous example also introduces a new, previously unseen method, quote(). This particular method is used to escape values that are passed to the database system, preventing SQL injection. This is PDO’s equivalent of mysql_real_escape_string() for MySQL, and pg_escape_string() for PostgreSQL, and so on. The underlying functionality is provided through the database’s native escaping mechanism. If one is not available, then a boolean FALSE is returned, indicating that prepared statements should be used instead of direct query execution, more on that in a bit. Result Iterator One of the neat tricks that is possible with PDOStatments is the ability to iterate through it as if it was an array of
30
FEATURE
An Introduction to PDO
results, via the foreach construct, thus avoiding the need to call any functions and methods, which of course leads to greater performance. The iterator approach represents the fastest mechanism of retrieving data—even exceeding that of fetchAll()—since it does not require the pre-fetching of all results, and their subsequent storage in memory. foreach ($db->query(“SELECT id FROM foo”) as $v) { // $v == array(0=>1, ‘id’=>1) }
One “limitation” of this approach to data retrieval, is that there is seemingly no way to indicate the fetch mode. PDO does provides a solution for this, it’s just not as obvious as with the other fetching methods. The workaround involves passing the fetch mode via the second optional parameter of the query() method. foreach ($db->query(“SELECT id …”, PDO_FETCH_COLUMN) as $v) { // $v == 1 }
columns and PHP variables which will be populated with the relevant values. $stmt = $db->query(“SELECT login, pass FROM user”); $stmt->bindColumn(1,$login); $stmt->bindColumn(2,$pass);
The bindColumn() function, at its very minimum, requires two arguments. The first is the numeric position of the column to which you’d like to bind the variable. This position, interestingly enough, starts the count at 1, rather than 0. This is a bit of an inconsistency when it comes to PHP, and even other parts of PDO that normally start at 0. For better or worse, the developers have decided to follow the approach used by other similar interfaces in other languages, so be careful. If using the numeric position of the column seems like too much of a pain, due to this inconsistency, you can, of course, use the name of the column as a point of reference. However, to do this, you need to know the case of the column name. If the returned column is “FFOO”, and you try to bind “ffoo”, the association
“One of the tricks that is possible with PDOStatments is the ability to iterate through them as if they were arrays of results, via the foreach construct.” Now, instead of getting a complicated array, each $v variable (representing a row) is a simple and immediately-usable string value. This mechanism supports all of the same modes as the ones supported by fetch(). To make things even more interesting, setting the fetch mode inside the query() method saves you from having to specify it inside the fetch() calls, or having to explicitly call the setFetchMode() method, making the code a bit simpler. Parameter Binding Another approach to data retrieval involves variable binding. In this case, rather than creating a new variable with an array or object container being for every record, an existing variable or variables are automatically populated with returned information. This approach can be quite handy in many situations, such as template population, for example, when in most instances the retrieved data needs to be assigned to template variables. With variable bindings, this can be done completely automatically, simplifying the code and, in some instances, also improving its performance. To use variable bindings, the bindColumn() method of the PDOStatment object needs to be used, prior to data retrieval, to create associations between the result
September 2005
●
PHP Architect
●
www.phparch.com
process will fail. The second parameter to the function is much simpler; it is simply the variable whose value will be populated by the fetch process. As you can imagine, bindColumn() takes this value by reference. $stmt = $db->query(“SELECT id,login FROM user”); $stmt->bindColumn(‘id’, $id); $stmt->bindColumn(‘login’, $login); while ($stmt->fetch(PDO_FETCH_BOUND)) { // $login == current value of login column (as a string) // $id == current value of id column (as an integer) }
When it comes to fetching the data, the PDO_FETCH_BOUND mode is passed to the fetch function, which ensures that the returned value is not a variable with data, but merely a boolean indicator used to determine if further records are available. The data itself will, of course, be available through the bound variables, whose values will be appropriately adjusted on every successful fetch. Partial Data Retrieval When it comes to data retrieval, PDO tries to take the most memory efficient approach possible. This involves the use of unbuffered queries that does not require prefetching of the complete result set into memory. 31
FEATURE
An Introduction to PDO
The consequence of this optimization is that pending results remain active on the connection until they are retrieved. If an attempt is made to execute another query, prior to the retrieval of all records returned by the previous operation, an error condition will be triggered. To keep this issue from becoming a real problem, the script could forcibly retrieve all rows by executing while($stmt->fetch()); to ensure that no rows are left over. However, that would be highly inefficient, as it would result in the retrieval and temporary storage of unnecessary data. A much easier, and far more efficient solution involves the use of the LIMIT clause to limit the result set to the subset you intend to use. If that is not possible, then PDO provides the closeCursor() method, which can be used to forcibly terminate a result set that has not yet been completely retrieved. $stmt = $db->query(“SELECT * FROM foo”); while ($res = $stmt->fetch()) { /* some code that may set $abort to TRUE */ if ($abort) { $stmt->closeCursor(); break; } } // now can safely execute another query.
Prepared Statements There is yet one more feature of PDO that has, so far, been neglected, and deserves a mention. One of the core capabilities of PDO is the ability to use prepared statements, regardless of native database support for this functionality. Prepared statements are a very interesting bit of functionality as they increase both the security and the performance of an application. Prepared statements work by allowing separation between the parsing of the query and its execution. For example, when the query() method is used to execute dynamic SQL, every instance of a query call involves the database parsing the query and then executing it. While query parsing process is quite fast, if you end up executing the same (or similar) query multiple times, it does make things somewhat inefficient. With prepared statements, on the other hand, the query is pre-parsed, leaving places for dynamic tokens. The generated statement can then be reused multiple times. The execution step now merely needs to substitute the tokens with the given values, effectively eliminating all but one query parsing operation. The security advantage comes from the fact that tokens are no longer treated as part of the query—as is the case with dynamic query execution—and will always be interpreted as a value and nothing more. This means that SQL injection is no longer a possibility, and you don’t need to escape the input using the quote() method, which provides an extra bit of performance. Making use of prepared statements in PDO is a fairly simple process that consists of just two steps. The first
September 2005
●
PHP Architect
●
www.phparch.com
step is the compilation of the given SQL query into a statements, via the prepare() method. $stmt = $db->prepare(“INSERT INTO foo (a,b) VALUES(?,?)”);
If the query contains variable values, they can be represented using the “??” character or assigned by name, which may make the query a bit easier to understand. $stmt = $db->prepare(“INSERT INTO foo (a,b) VALUES(:a, :b)”);
Upon successful query parsing, a PDOStatment object will be returned that can then be used to execute queries based on the previously compiled SQL. This is accomplished via the execute() method, which takes an optional parameter—an array of values to be substituted into the dynamic tokens of the compiled query. As you can probably imagine, the number of values in the array must match that of the dynamic tokens in the query. If the unnamed tokens are used, then the parameter array should be a simple, and one-dimensional, where every single element corresponds to a token. $stmt->execute(array(1,2));
On the other hand, if named tokens were used, then the complete token name, including the “::” character should be used as an associative array key that points to the desired value. $stmt->execute(array(‘:a’=>1, ‘:b’=>2));
The dynamic tokens can be bound to variables, so that the array need not be created and passed via execute(), each time. This can be particularly useful, if the data is coming from another source such as a CSV file, and is already broken down into variables. $stmt = $db->prepare(“INSERT INTO users (name, email) VALUES(?, ?)”); $stmt->bindParam(1, $name, PDO_PARAM_STR, 255); $stmt->bindParam(2, $login, PDO_PARAM_STR, 32);
The bindParam() method associates a variable to a particular dynamic token, by using the token’s position. Once again, counting starts at one rather then zero. For each bound variable, we specify a type, based on the PDO_PARAM constant, which tells the database how to treat the input data. For strings, it is also possible and recommended to specify the maximum length of possible values to facilitate internal optimizations, for various database systems. Once the variables have been bound, their values can now be populated from the CSV file, and subsequent execute() method calls can be used to insert these values into the database. $fp = fopen(“users.csv”, “r”); while ($csv = fgetcsv($fp, 1024)) { list(,$name,$email,) = $csv; $stmt->execute(); }
32
FEATURE
An Introduction to PDO
The use of prepared statements is not limited to a particular query type. With the exception of table creation and modification queries, nearly every other query can be made into a prepared statement. In the case of SELECT, the same fetch process as we’ve seen before (at the completion of the execute()) can be used to retrieve the data. Not all databases support prepared statements, and in some instances their support may only supply the part of the internal support required by PDO. In those cases, an emulation layer that is built-in to PDO will be used to replace the absent features of the database in question. For example, in the case of the MySQL driver, prepared statements are only available in version 4.1.3 or later, and therefore will be emulated for older releases. In some situations, such as in the case of PostgreSQL, the database may support the functionality natively, this support is poor. When using native prepared statements PostgreSQL sometimes fails to optimize the query properly, leading to slower execution time. For this reason, the PDO_PGSQL_ATTR_DISABLE_NATIVE_PREPARED_STATEMENT attribute was added, which, if enabled, makes the driver use PDO’s emulation layer, rather than the native functionality. Utility Functions Aside from the previously mentioned mechanisms, PDO also provides a number of utility methods to facilitate various operations. These include PDOStatment’s columnCount() method, which will return the number of columns inside a result set. It can be coupled with the getColumnMeta() method of the same object to retrieve information about the contents of a particular column (provided by the database), including some PDO-specific information that is available for all database drivers.
• precision – the numeric precision of this column • pdo_type – the column type according to PDO, as one of the PDO_PARAM constants Another useful PDO utility function is yet another PDOStatment method, rowCount(). For databases that use buffered queries, this method can be used to determine the total number of rows found in the result set. In the event that the database does not support this functionality, or where unbuffered queries are being used, the size of the result set is not known, and the returned value will be -1. $db->query(“SELECT * FROM users”); $db->rowCount(); // returns -1, since unbuffered queries are used by default $db>setAttribute(PDO_MYSQL_ATTR_USE_BUFFERED_QUERY,1); $db->query(“SELECT * FROM users”); $db->rowCount(); // return a value row count, since the query is now buffered.
Additional utility and database-specific functions may be added in the future, and if you have an idea for a generally useful PDO function, feel free to voice your suggestion at http://bugs.php.net/, via a feature request, or on the PHP-Internals mailing list. Either way, suggestions—or better yet, patches—are more than welcome. Incidentally, this concludes our brief tour of PDO and its functionality, which hopefully convinced you to consider PDO as the interface for your next project.
$c = $stmt->columnCount(); for ($i=0; $i < $c; $i++) { $meta_data = $stmt->getColumnMeta($i); }
As you’ve probably guessed, the enumeration of columns starts at zero, following the standard PHP convention. The returned value is an associative array containing the following data set: • native_type – the PHP data type • driver:decl_type – the data type of the column, according to the database • flags – any flags particular to this column, in array form • name – the name of the column, as returned by the database, without any normalization • len – maximum length of a string column; may not always be available, and will be set to -1 if it isn’t
September 2005
●
PHP Architect
●
www.phparch.com
About the Author
?>
Ilia Alshanetsky is the principal of Advanced Internet Designs Inc., which specializes in security auditing, performance analysis and application development. He is the author of FUDforum (http://fudforum.org), a highly popular, Open Source bulletin board, focused on providing the maximum functionality at the highest levels of security and performance. Ilia is a core PHP Developer, an active member of PHP’s QA team, and was the Release Master for the PHP 4.3.x series. He has authored and co-authored a number of extensions, most notably SHMOP, PDO, SQLite and GD, and is responsible for a large number of bug fixes and performance tweaks in the language. A prolific lecturer and writer, Ilia can found speaking at international conferences. He is frequently published in print and online magazines on a variety of PHP topics, and is also the author of an upcoming book on PHP security. Ilia can be reached at
[email protected].
To Discuss this article: http://forums.phparch.com/249
33
NEXCESS.NET Internet Solutions 304 1/2 S. State St. Ann Arbor, MI 48104-2445
http://nexcess.net
PHP / MySQL SPECIALISTS! Simple, Affordable, Reliable PHP / MySQL Web Hosting Solutions P O P U L A R S H A R E D H O S T I N G PAC K A G E S
MINI-ME
$
6 95
SMALL BIZ $ 2195/mo
/mo
500 MB Storage 15 GB Transfer 50 E-Mail Accounts 25 Subdomains 25 MySQL Databases PHP5 / MySQL 4.1.X SITEWORX control panel
2000 MB Storage 50 GB Transfer 200 E-Mail Accounts 75 Subdomains 75 MySQL Databases PHP5 / MySQL 4.1.X SITEWORX control panel
16 95
/mo
900 MB Storage 30 GB Transfer Unlimited MySQL Databases Host 30 Domains PHP5 / MYSQL 4.1.X NODEWORX Reseller Access
NEXRESELL 2 $
We'll install any PHP extension you need! Just ask :) PHP4 & MySQL 3.x/4.0.x options also available
59 95
/mo
7500 MB Storage 100 GB Transfer Unlimited MySQL Databases Host Unlimited Domains PHP5 / MySQL 4.1.X NODEWORX Reseller Access
: CONTROL
php 5 4.1.x
POPULAR RESELLER HOSTING PACKAGES NEXRESELL 1 $
NEW! PHP 5 & MYSQL 4.1.X
PA N E L
All of our servers run our in-house developed PHP/MySQL server control panel: INTERWORX-CP INTERWORX-CP features include: - Rigorous spam / virus filtering - Detailed website usage stats (including realtime metrics) - Superb file management; WYSIWYG HTML editor
INTERWORX-CP is also available for your dedicated server. Just visit http://interworx.info for more information and to place your order.
WHY NEXCESS.NET? WE ARE PHP/MYSQL DEVELOPERS LIKE YOU AND UNDERSTAND YOUR SUPPORT NEEDS!
php 4 3.x/4.0.x
128 BIT SSL CERTIFICATES AS LOW AS $39.95 / YEAR DOMAIN NAME REGISTRATION FROM $10.00 / YEAR GENEROUS AFFILIATE PROGRAM
UP TO 100% PAYBACK PER REFERRAL
30 DAY MONEY BACK GUARANTEE
FREE DOMAIN NAME WITH ANY ANNUAL SIGNUP
ORDER TODAY AND GET 10% OFF ANY WEB HOSTING PACKAGE VISIT HTTP://NEXCESS.NET/PHPARCH FOR DETAILS
Dedicated & Managed Dedicated server solutions also available Serving the web since Y2K
FEA T URE
FEA TURE
What Are
Trackbacks And Why Do They Exist by Chris Cornutt
If you’ve been around the internet for any length of time, chances are you’ve seen a weblog. Chances are, if you’ve seen a weblog, then you’ve seen a trackback. You might not have known it at the time or even understood what it was, but more and more of the blogging tools out there are using them. So, what are these elusive trackbacks and why do they even exist?
B
ack in August of 2002, a group called Six Apart (creators of the Movable Type weblog system) decided that there needed to be a way for one blog to inform another when linking to it. Sure, the administrator of the linked blog could just look at their web server logs and see where the hits were coming from, but trackbacks offer a dynamic way—for not only the site’s owner but also other visitors to the page—to see how many other sites had linked to them. These links could include anything from the URL of the linking site, to the site’s name, or even a snippet of the story from which it came. Six Apart created a technical specification for the transmission of these “pings,” back and forth between sites, and published it. Shortly after that, in October of 2002, they made some modifications to the specification, changing things like the protocol to use when sending trackbacks, and implemented some of the auto-discovery options. So, with spec in hand, several
September 2005
●
PHP Architect
●
www.phparch.com
REQUIREMENTS PHP
n/a
CODE DIRECTORY
trackbacks
RESOURCES URL
htt p:/ /ww w.i onc ube .co m/
URL
htt p:/ /ww w.z end .co m/
URL
htt p:/ /ww w.s our ceg uar dia n.c om/
URL
htt p:/ /ww w.p hpa udi t.c om/
i
of the existing blogging tools set out to implement this handy notification mechanism. They envisioned sites automatically linking to other sites, links upon links, dynamically relating blog content pages to one another. Some of the tools that have worked trackbacks into their structure include Movable Type (obviously), WordPress, Radio, and Serendipity. Unfortunately, as is usually the case with any kind of automatic resource that’s put out in the public view, people have seen fit to abuse trackbacks, filling peo-
37
FEATURE
Trackbacks
ples’ pages with random links and other such spam. Of course, bloggers are no strangers to spam on their pages, as comment spam is a prevalent problem these days, as well. Thankfully, there have been several efforts to help squelch the spam problem in the form of PEAR classes and other independent projects. Most of the popular blogging tools, however, don’t have built-in support for things like this. They do allow you to remove the offending trackbacks, easily, but when you have hundreds coming in a day (yes, it happens) that’s just not practical. Support for this kind of filtering is getting better, though—the WordPress blogging software, for example, has made a large, concerted effort to integrate filtering into their code. Tobias Schlitt has created a PEAR class that seeks to help with the problem too: Services_Trackback. This package provides all of the basic functionality that someone who would like to implement trackbacks on their site needs: sending a trackback, receiving a trackaback, autodiscovery of trackback URLs, etc. Where it really shines, though, is in the filtering techniques that it employs. There’s so much more than just the simple word list filtering here; it also offers regular expression matching, a DNS blacklist option, and sub-URL matching. Other bloggers go the other route, however, and simply turn trackbacks off, completely. They either don’t have the time to worry about the filtering or just don’t want the hassle of having to deal with them. Of course, to others, they simply seem like glorified links, serving no other purpose than to allow someone else to shamelessly promote themselves on someone else’s blog. Some users out there see the idea of trackbacks as somewhat rude, allowing anyone and everyone to post pretty much whatever they want to an entry of yours. It’s pretty easy to see how the possibility for spam and abuse wouldn’t be far behind this one. With the protocol open to anyone, and no inherent security for the posts, there’s really not much stopping someone who wants to spam their message all over your pages. Granted, some of the blogging software out there does its best to try to limit the spam that’s received, but with automation of the trackbacks being such an easy thing, it’s almost not worth it, at times. Some bloggers have left trackbacks on for a while, only to be burned by a spammer coming into their site and abusing its trackback interface. Of course, the use of trackbacks isn’t limited to just weblog software—there are people that have stretched the use of this handy little protocol to make it do other things for them. One example comes from the weblog of Matthew Haughey, founder of Metafilter.com, in which he actually posts a “Now Playing” item to his site from either Winamp or iTunes. He has scripted an interface from these two pieces of software through a trackback-style interface and, with the help of DoSomething/AppleScript, he has been able to send the information to his site. The protocol is pretty open, September 2005
●
PHP Architect
●
www.phparch.com
Listing 1
1 function sendPing($data_array){ 2 $request=””; $content=””; 3 $url_parts=parse_url($data_array[‘tb_url’]); 4 $content.=(isset($data_array[‘title’])) ? 5 6 “title=”.$data_array[‘title’].”&” : “”; 7 $content.=”url=”.$data_array[‘url’].”&”; 8 $content.=”blog_name=”.urlencode($this>site_name).”&”; 9 $content.=”&”.$url_parts[‘query’].”\n”; 10 $request.=”POST “.$url_parts[‘path’].” HTTP/1.0\r\n”; 11 12 $request.=”Host: “.$url_parts[‘host’].”\r\n”; 13 $request.=”Content-Type: “; 14 $request.=”application/x-www-form-urlencoded; “; 15 $request.=”charset=utf-8\r\n”; 16 $request.=”Content-Length: “.strlen($content).”\r\n”; 17 $request.=”\r\n”; 18 $request.=$content; 19 20 echo “<pre>request:
”.$request.””; 21 $response=$this->socket($request,$url_parts[‘host’]); 22 echo “”; 23 echo “<pre>response: “.htmlspecialchars($response). 24 “”; 25 }
Listing 2 1 function socket($request,$host){ 2 $string=””; 3 $fp=fsockopen($host,”80”,$errno,$errstr); 4 if($fp){ 5 fwrite($fp,$request); 6 while(!feof($fp)){ 7 $string.=fread($fp,1024); 8 } 9 fclose($fp); 10 }else{ echo “Error: “.$errno.”: “.$errstr.”
”; } 11 return $string; 12 }
Listing 3 1 function handlePing(){ 2 $contents=””; 3 $fp=fopen(“php://input”,”r”); 4 while(!feof($fp)){ $contents.=fread($fp,1024); } 5 fclose($fp); 6 $parts=explode(“&”,$contents); 7 foreach($parts as $key => $value){ 8 if(!empty($value)){ 9 $p=explode(“=”,$value); 10 $arr[$p[0]]=$p[1]; 11 } 12 } 13 return $arr; 14 }
Listing 4 1 80 81
September 2005
●
PHP Architect
●
www.phparch.com
and can be adapted to more uses than just the typical commenting and linking done on most blogs. The real potential behind trackbacks can be seen in the fact that a trackback is more than just a normal link to another page. It’s more of a meta-link, providing more information than just a referrer in your server’s web logs. If you have a site that doesn’t currently have (or can’t really use) trackbacks, you might consider one of the trackback “hosting” services such as HaloScan.com. They offer a service that, with “just two lines of code”, can offer you all the benefits of having trackbacks on your site. They offer services like the banning of commenters, CSS templating, and a custom RSS feed for the trackbacks/comments left on your site. The How So, now that we’ve talked about how trackbacks can be used, and their potential for abuse (unfortunately), how can we actually create these handy little “pings?” Well, thankfully, the protocol is a very simple one as shown on the Six Apart pages. Trackbacks use a REST model (Representational State Transfer) and are created the same way that a normal HTTP call is performed. The request consists of a formatted POST request with certain variables set. Only one of these is required, the URL that the ping is coming from, and the rest are optional, but helpful—the title of the entry, an excerpt from the page, and the originating blog name, for reference. Each kind of weblog system that I looked into seemed to have its own interface for accepting trackbacks, with none of them resembling each other in name or URL to call. A formatted request looks like this: POST http://www.example.com/trackback/5 Content-Type: application/x-www-form-urlencoded; charset=utf-8 title=Foo+Bar&url=http://www.bar.com/&excerpt=My+Exce rpt&blog_name=Foo
It’s a normal HTTP POST request to the URL specified by the original entry, with a specific Content-Type, and the data encoded in the typical POST format. According to the specification for the request format, you must send a Content-Type header (set to “aapplication/x-wwwform-urlencoded; charset=utf-8”), in order for the request to be successfully accepted. Of course, the formatting of the values in the data section of the request must conform to the character set that you’ve indicated. If the request was successful, the trackback script should respond with an XML response: <error>0
On success, the error tag will contain a “0”. Otherwise, it will contain an error message such as “We already
39
Any more, and we’d have to take the exam for you! We’re proud to announce the publication of The Zend PHP Certification Practice Test Book, a new manual designed specifically to help candidates who are preparing for the Zend Certification Exam. Available in both PDF and Print
Written and edited by four members of the Zend Education Board, the same body that prepared the exam itself, and officially sanctioned by Zend Technologies, this book contains 200 questions that cover every topic in the exam. Each question comes with a detailed answer that not only provides the best choice, but also explains the relevant theory and the reason why a question is structured in a particular way. The Zend PHP Certification Practice Test Book is available now directly from php|architect, from most online retailers (such as Amazon.com and BarnesandNoble.com) and at bookstores throughout the world.
Get your copy today at http://www.phparch.com/cert/mock_testing.php
FEATURE
Trackbacks
have a trackback from that URL on this post.” Basically, when a given user posts a new note on his blog, he can enter a trackback URL for the entry that he’s posting about. His blogging software then takes this URL (and possibly some other information about the entry) and creates a POST request destined for it. One thing to look out for when posting a trackback: the URL for the trackback interface is usually different from the post being linked to. This URL can usually be found somewhere on the destination page, usually right between the comments and the main body of the post. The software will then send the request to this URL and the remote blog’s software will interpret it. There are many reasons that a post could be denied, one common error occurs when you’ve already posted a trackback to the target entry. There is also a provision in the trackback specification that allows for the auto-discovery of the trackback URL for a given post, but the weblog software has to support this feature. The method that a client uses to autodiscover the trackback URL for each post involves looking at the RDF/RSS file for the site. Most of the software that’s out there automatically creates a syndication file for your site, allowing people to subscribe with their aggregators and see when you make a new post. This same syndication file is used to share the trackback URL with your visitors. There is metadata placed in each entry of the RDF (for each post) with the trackback namespace:
Note the “ttrackback:ping” URL that’s provided there at the end. That’s what your script would need to look out for. Unfortunately, as I mentioned, it doesn’t see like many sites really use this format, so you might be stuck with parsing the page in an effort to find it. Thankfully, just about every page I’ve seen that has trackbacks enabled used the word “trackback” in the link for it on the post. Grabbing the page and parsing out this URL isn’t too big of a problem—as is evidenced, unfortunately, by the rampant trackback spam that goes on. The Code So, the formatting is great and all, you say, but when do we get to the actual code? Well, ask and ye shall receive. I’m going to show you a little class that I whipped up to deal with the sending and receiving of trackbacks. This is a very simplified version of something that could get really complex really quickly, so don’t expect much more than a simple send/receive. I’ll give you the code, then walk you through what it’s doing, line by line. The sendPing function (Listing 1) does just what it
September 2005
●
PHP Architect
●
www.phparch.com
sounds like: sends a trackback ping to another site. The $data_array that’s passed in contains information, from a form submission in this case, and can have the following values: tb_url, title, and url. Remember, the URL parameter is the only one that has to be set. If it’s not, chances are your request will be rejected. After initializing the $request and $content variables, I break out the parts of the trackback URL (ttb_url) that’s been given with parse_url(). The output of this function gives me all the data about the URL, including the path of the script, the host it’s being sent to, and the query that was on the end. It returns more than that, but for our purposes, that’s all we need. Then, we start building the content of the request— the data, not the headers. The $content variable is build according to the Six Apart specification. Once we have the content, we can build the actual request, in the $request variable. We needed to make the content of the request first so that we could use the strlen() function to give us a “CContent-Length”. The value of $request is appended to, and a normal POST request (with the required Content-Type) is created. Add a final line with the content itself and your request is complete. My code echo()s the $request for debugging purposes—so I could see what it was sending. After that, there’s a call to the $this->socket() function— this is another function in the class (Listing 2). The socket() function take the hostname and the data for a request. It’s called with the request that we’ve created, as well as the hostname pulled from the URL that sendPing() was given. A socket is opened to port 80 on the remote host and, if this operation is successful, the request is posted to the remote script. Once the request has been made and sent, the script looks for a response with fread() and, of found, appends it to the $string variable. The connection is closed and $string is returned to the sendPing() function. If there is when opening the socket, it will be passed to the else clause, and the error will be displayed. If all has gone well, your trackback (with the data you specified) should be on the page you submitted it to. Having the echo() statements in the sendPing() function can really help in the debugging process—sometimes web servers don’t return what you think they should. Ideally, though, the request that I laid out should work with any server out there. When it comes to receiving a ping, you want to do just the opposite—your script needs to look for a POST request from another page. Since most sites will have a separate script to handle their trackbacks, you could simply rely the assumption that any data posted to your trackback script is, in fact, a trackback request. Of course, that seems a little scary to me, so I threw an if on my page to check if $_POST[‘url’] was set. Since this is a required field for the trackback call, I figured that this is an acceptable check.
41
FEATURE
Trackbacks
Now, for the functionality that handles the ping, let’s take a look at Listing 3. First, we initialize the $contents variable that we’ll use to grab the contents of the POST request. Then, using the special “pphp://input” protocol in fopen(), we can grab everything that was fed to the script. When PHP is running in the
Trackback URL” field and the script should pick it up just fine. When the form is submitted, you’ll see the POST request that it sends, in the output, along with a response from the same script, eliminating the need to have two scripts for testing. When the POST request comes in, the script ensures that the “uurl” value is set,
“Trackbacks can include anything from the URL of the linking site, to the site’s name, or even a snippet of the stories from which they came.”
context of a HTTP request, this will grab all of the data except for the headers. There is also a $GLOBALS[‘HTTP_RAW_POST_DATA’] variable that can be accessed to get the same kind of information, but this variable is only populated when the server’s always_populate_raw_post_data ini directive is true. The POST data is appended to the $contents variable, inside the while loop, and the socket is closed. To get the values from the POST data into something we can use, we explode on the “&&” characters, which are used to separate the values in a POST request and put those into an array, $parts. Then, with a foreach loop, we go through each of the entries, exploding them on the “==” character (separating the key from the value) and set those in the $arr array. What we end up with it something like this: url=http://www.test.com&title=my%20title&blog_name=mi ne // becomes: [url] =>
http://www.test.com
[title] => my title [blog_name] => mine
This value in the $arr variable is then passed back out of the script to be used elsewhere. One thing that we’re not really going to look at, however, is auto-discovery of the trackback URLs. I’ve included an example of how to possibly parse out a trackback URL, along with the full class code in Listing 4, so you can see one possibility of how to this mechanism. The example uses a simple form to post the information through the trackback URL, and is set up in a manner where you can post the trackaback to the same page, and it will understand how to handle it. For example, if you named the page “ffoo.php”, once you created the class instance and have the form ready, you could put http://www.mysite.com/foo.php in the “Their
September 2005
●
PHP Architect
●
www.phparch.com
and that the “ttb_submit” field is not—this differentiates a trackback call from a regular (non-trackback) form submission. That’s pretty much it—using three simple functions, you have a base PHP class to help you send and receive trackbacks. As I said earlier, it’s nothing fancy and there’s no filtering, or other special features, in there; it’s just an introduction to the format and will send and receive. From here, you can add on whatever features you like—filtering, better error checking, etc. In Summary Trackbacks were designed to be a simple thing from the start, a friendly way to let other site owners know that you found their content interesting, and have linked to it in a post of your own. They can be very useful when used in the right manner—they can provide a meta-link from site to site, helping to link content and share ideas among a site’s visitors. The goal of articles referencing other articles, referencing other articles, etc. is a decent one, unfortunately, too many people out there have seen fit to abuse this simple “heads up” from one blogger to another. It’s all too common to come across posts that talk of problems with spam in trackbacks, or even stories of bloggers who have shut them off, altogether. It’s not too hard to tell when a site has been hit by a trackback spammer, either—the numbers don’t lie; a posting with a large number unrelated trackbacks is not hard to find. Since comments and trackbacks often occupy the same areas on most of the blogging tools out there, comment spam and trackback spam seem to go hand in hand. There are those out there that argue also that trackbacks, in themselves, really aren’t worth much to the online community. They argue that if you want to say something to the person posting on the blog, you
42
FEATURE
Trackbacks
should just leave a comment. Trackbacks are the comment equivalent of a PostIt Note. Their reasons range from the obvious spam difficulties all the way to is described as “uselessness.” Some bloggers suggest that trackbacks really aren’t needed—that a normal link to an entry is all that’s really necessary. They point out that trackbacks aren’t centralized enough to do any kind of good. Yes, they link back and forth between blogs, helping to bridge the gap that the user would otherwise have to search for, but since they are unique to each weblog and aren’t really stored in any centralized manner, they’re not really useful for much more than a “look at me! I linked to you!” sort of message. Of course, logically following that argument, the topic of Technorati is brought in. Technorati is a site (http://technorati.com/) that indexes and mines the data from millions (15.5 million at the time of writing) of weblogs. The site includes the ability to search all of the content, and show only results pertinent to your search. Many of the developers that see trackbacks as pointless seem to think that Technorati is a much better solution. Not only does the site index the content to make it searchable, but using their search feature, you can see which blogs have linked to the site of your choosing. The results include the name of the site, a link to it, the brief bit of content surrounding the search term, and how long ago the item was posted. They’ve even introduced an API that you can connect to, to perform these searches right on your own page (but using it’s a topic for another time). For example, a visitor could come to your site, view one of the postings on your blog, and see, courtesy of Technorati, an up-tothe-minute list of sites that link to that very page.
September 2005
●
PHP Architect
●
www.phparch.com
Sound familiar? Well, it should—and using this functionality, Technorati could very easily replace trackback functionality for any site out there. All they’d need is access to the API, one of the many libraries out there for accessing it, and a bit of patience to get it up and running. Trackbacks, on the whole, seem to be sticking around for a bit longer, but with things like the Technorati example above, it’s only a matter of time before they get phased out. Sure, you can argue the old “what happens if they just go away” mentality that applies to so many situations, but the benefits of such a centralized system really make linking between blogs much more useful. In the mean time, though, Trackbacks are still the de facto method of letting a blog’s maintainer know that you’re linking to her content.
About the Author
?>
Chris has been involved with PHP and its community for about five years now, most of that running his site, PHPDeveloper.org - a site devoted to bringing the most up-to-date, informative news and community happenings to the forefront. He’s a Zend Certified Engineer and works as a web site administrator at a large natural gas utility in Dallas, Tx.
To Discuss this article: http://forums.phparch.com/250
43
FEATURE
End-to-End Testing with
PHP and Internet Explorer by Oz Solomon
Automated testing can greatly improve the quality of your product. In this article, Oz presents a framework for creating automated tests that can simulate end-user activity. By leveraging the full faculty of Internet Explorer, these tests can do just about anything that your users can do.
A
few years ago, my team was suddenly pulled away from the project it was working on, and was commissioned to write a new transaction processing system for our company. Despite the complexities and strict reliability requirements imposed on the system, I was not given enough of a budget to hire any QA staff, let alone purchase expensive testing software. Fortunately, I manage a team of extremely bright people, and together, we were quickly able conquer the reliability beast. We were able to consistently keep a schedule that called for new releases of our system every 3-4 weeks. The level of reliability and maintainability that was required for such a tight schedule to be realistic was achieved due to our extensive use of automated tests. In this article, I will present a framework similar to the one that we developed to perform end-to-end testing of our web applications. This framework uses nothing more than PHP and Internet Explorer, and can be used to test anything Internet Explorer can get its hands on, be it PHP pages, JSP pages or hand coded HTML. Unit Tests vs. End-To-End Tests Before I discuss the framework, allow me to quickly review the differences between unit tests and end-toend tests. A unit test is a piece of code that exercises one function or functional unit and ensures that it works correctly. For example, if you write a function, validateCreditCardNum(), that checks for valid credit
September 2005
●
PHP Architect
●
www.phparch.com
REQUIREMENTS PHP
4.3.2+ (except 4.3.10), 5.x
OS
Microsoft Windows
Other Software
Internet Explorer, PHPUnit (PEAR)
Code Directory
endtesting
card numbers, the function’s unit tests would exercise it with various credit card numbers, ensuring that the returned values are correct on every call. In contrast, end-to-end tests exercise multiple components or the entire product to ensure that all the pieces work properly together. For example, imagine that you have a web form that accepts a credit card number. That form then submits to a PHP script that validates the credit card number and does some other processing. An end-to-end test would submit valid and invalid credit card numbers through the form, checking for proper responses from the application. Notice that you are no longer checking the validateCreditCardNum() function by itself. You are now checking the application as a whole. By running these tests, you will know, indirectly, that: • Your application is properly submitting the form • The backend code is properly passing credit
44
FEATURE
End-to-End Testing
card numbers to the validation function • The validation function is working as expected • The application is properly rendering errors Depending on your configuration, even the simplest end-to-end test can exercise many components, such as database layers, caching systems and graphics
Listing 1 1 16
Listing 2 1 15
Listing 3 1 23
engines. In an application of any size, there is a place for both unit tests and end-to-end tests. The discussion of when to choose one over the other is not within the scope of this article. Instead, I focus on how to actually write end-to-end tests once you have determined that they are appropriate for your project. The Goal The goal is to be able to write simple test scripts in PHP that control Internet Explorer. IE will then download pages on our behalf, populate web forms, click on buttons and even execute JavaScript. The advantage of using Internet Explorer as opposed to accessing the web pages directly via HTTP is that there is no need to parse HTML or simulate JavaScript. A fully functional browser will be under your control. In fact, since 90% of web users employ Internet Explorer as their primary browser, tests that use IE will very closely match the end-user experience. You should still test your site with other browsers (I personally use Firefox and ensure that all my sites are compatible), but after running your testsuite using IE, you can constrain your Firefox testing to cross-browser compatibility issues, not application issues. For the test framework to be successful, it must be able to perform basic operations with very few lines of code. The following code is representative of the type of script that we want to be able to write: // load a web page $doc = $this->loadUrl(‘…’); // get a reference to an element $elem = $doc->getElementById(‘foo’); // make sure that the element has the correct value $this->assertEquals(‘bar’, $elem->value, “foo’s value”);
The assertEquals() method that was used in the code above is part of the PHPUnit framework from PEAR ( http://pear.php.net/package/PHPUnit ). It ensures that that the second parameter (the tested value) is equal to the first parameter (the expected value). I will be using PHPUnit as the foundation for the end-to-end testing framework. You should familiarize yourself with PHPUnit by reading its documentation at http://pear.php.net/manual/en/packages.php.phpuni t.intro.php.
Hosting Internet Explorer I will use PHP’s COM support to create and control an instance of Internet Explorer. A COM extension for the Windows version of PHP has existed for quite some time, and using it couldn’t be easier. To create a COM object (in our case an Internet Explorer application object), all you need to do is create an instance of the COM pseudo-class: $ie = new COM(‘InternetExplorer.Application’)
September 2005
●
PHP Architect
●
www.phparch.com
45
FEATURE
End-to-End Testing
Listing 6
With this new variable in hand, we can start manipulating Internet Explorer. First, we can make it visible: $ie->visible = true;
Notice that you will only see the Internet Explorer window if you are running your PHP script from the command line, or if your web server is not running as a Windows service. When PHP is running inside a service (which is mostly the case when you are accessing PHP scripts through a web server), it cannot interact directly with the desktop, and neither can the objects that it spawns. To load a web page, call the Navigate2 method: $ie->Navigate2(“http://www.php.net”);
If you run Listing 1, which brings the above-mentioned concepts together, you will notice a small problem: the script exists, but Internet Explorer remains active. Clearly, this is not what we want. The solution is to ask Internet Explorer to exit using its Quit method, as shown in Listing 2. Once a page is loaded into the browser, you can access its contents using Internet Explorer’s document object. This object is equivalent to the document object that’s exposed to JavaScript, and shares all the same methods. For example, to access the entire document’s HTML source, use: Listing 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Listing 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
<script> function onButtonClick() { document.location = “http://www.php.net”; }
September 2005
●
PHP Architect
●
www.phparch.com
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Listing 7 1
Listing 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Listing 9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Go to Listing 10 without passing a value.
Value to pass to Listing 10:
September 2005
●
PHP Architect
●
www.phparch.com
To see more uses of the document object, take a look at Listing 3, which will load the php.net homepage, and then extract the list of headlines displayed there. It begins by loading http://www.php.net/ using Navigate2() on line 6, then goes on to obtain the page’s HTML source (lines 12-13). With the HTML and a small regular expression at hand, the list of headlines is easy to extract (line 17). So far, we haven’t done anything really exciting. In fact, if all we wanted to do was download a page and parse its HTML, then an HTTP library such as Curl would have served the purpose. Unfortunately, curl would fall short if we wanted to do anything more sophisticated. We have a full-blown browser at hand and we can use it to actually perform actions on the HTML page. Listing 4 demonstrates how a PHP script can click on a web form button. As you can see in Listing 5, the button has a JavaScript event attached to it via the onClick attribute. The PHP-initiated button click will properly trigger that JavaScript event. The actual click is performed with just two lines: // find the button $btn = $doc->getElementById(‘idBtn’); // use the button’s built in click method to simulate a click $btn->click();
Waiting For the Page to Load Have a look at Listing 3 one more time. Specifically, notice line 9. This line pauses for one second to give Internet Explorer enough time to actually download the content of the web page. One second may seem like an arbitrary amount, and in fact it is. It works just fine on my machine, but won’t work so well for you if you are using a modem. In fact, many factors such as machine speed, network congestion and the load on the target server can all affect the amount of time it would take the page to download in full. Listing 6 introduces a new function, waitForDocumentReady(), whose responsibility is to determine if the entire web page has downloaded. Remember that a web page contains more than just HTML. External objects, such as images and flash animations must finish downloading before the page’s document is ready for manipulation. waitForDocumentReady() uses two different Internet Explorer properties to check for readiness. The first property, Busy , is exported by the browser application object (see http://msdn.microsoft.com/workshop/browser/webbro wser/reference/ifaces/iwebbrowser2/iwebbrowser2.a sp). The second is the readyState property, which is
47
FEATURE
End-to-End Testing
defined
by
the
document
object
(see
http://msdn.microsoft.com/workshop/author/dhtml/r eference/objects/obj_document.asp ). The function
loops and pauses until both of these indicators are in the proper state. waitForDocumentReady() will also detect if Internet Explorer was closed down behind our back. Since the browser window is visible and looks like any other application, anyone can close it at any time. This will cause the Busy property to return null, which in turn will cause waitForDocumentReady() to immediately terminate the PHP script. The Browser Object Listing 7 combines waitForDocumentReady(), as well as browser lifetime management mechanism, and some utility functions into a reusable Browser class. This class uses the singleton pattern. The reason I have made it a singleton is that I have noticed over the years that instantiating more than one Internet Explorer object within a single PHP script can cause intermittent problems, so it is best to allow only a single browser instance per PHP script. Listing 8 is similar to Listing 3, with the exception that it uses the Browser class. The class simplifies the process of waiting for the web page to load (the loadDocument() method automatically calls waitForDocumentReady()). Browser also takes care of closing Internet Explorer when the script exits (we no longer need to call Quit, explicitly). End-To-End Testing With BrowserEET_TestCase It’s time to write some automated tests. As previously
mentioned, I will use the PHPUnit framework as a foundation for all the tests. In PHPUnit, all test cases inherit from the PHPUnit_TestCase base class. I will derive a BrowserEET_TestCase class, which will serve as the base for all end-to-end tests. To see BrowserEET_TestCase in action, let’s create two pages, Listing9.html and Listing10.php. From Listing 9, you can submit a value to Listing 10, or go to Listing 10 directly through a link (in which case no value is passed). Listing 10 will then display the value passed from Listing 9. We will start with version (a) of Listing 10. Listing 11 contains some tests I wrote for Listings 9 and 10. It begins by setting up a “base” URL (line 14). Base URLs are a handy feature built into BrowserEET_TestCase which allow you to use relative paths to your test server. Usually, when testing, your scripts will reside on a staging server, or on your own machine. To avoid having to use the full path to all your pages (e.g. http://localhost:8080/page.php or http://stagingserver/build1830/page.php), you can set the base URL just once and afterwards, use relative URLs. You can still use absolute URLs to load pages located outside of the base. For example: $this->setBaseUrl(‘http://localhost:8080/phpa’); // load http://localhost:8080/phpa/listing9.html $this->loadUrl(‘/listing9.html’); // load http://www.php.net/docs.php; base URL // is ignored due to use of http:// $this->loadUrl(‘http://www.php.net/docs.php’);
The first test to execute is testGoThroughLink. This test simulates a click on the link found in Listing 9, and then ensures that Listing 10 generates the proper output. Right away, we see a problem:
Listing 10
Listing 10b
1
Listing 10a 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Welcome to Listing 10
The value of the day is... <span id=”idDisplayedValue”>
Back to Listing 9
September 2005
●
PHP Architect
●
www.phparch.com
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Welcome to Listing 10
The value of the day is... <span id=”idDisplayedValue”>
Back to Listing 9
48
FEATURE
End-to-End Testing
d:\prj>c:\php\cli\php Listing11.php D:\prj\BrowserEET.php(230) l…_9_10_tests::fail(...) D:\prj\BrowserEET.php(289) htmldocument::assertnophpscripterror(...) D:\prj\Listing11.php(29) l…_9_10_tests::loaddocument(...) … D:\prj\Listing11.php(75) phpunit::run(...) Test case failed: testgothroughlink Error message: PHP Notice Detected: Notice: Undefined index: val in D:\prj\Listing10a.php on line 3 url=http://localhost:8080/phpa/Listing10a.php
Oops! When writing Listing 10 (a), I assumed that $_GET[‘val’] is always valid, when in fact it is not. Thankfully, I have error_reporting=E_ALL set in my php.ini file (you should too when testing!), so PHP complained right away. The BrowserEET_TestCase::loadDocument() method always checks for script errors when it loads a page. The call
set, I use a preset value. This fix is enough to satisfy testGoThroughLink, which now completes without error. In fact, the second test, testGoThroughForm passes as well. Unfortunately, testGoThroughForm_UseTags fails: … D:\prj\Listing11.php(68) l…_9_10_tests::assertequals(...) D:\prj\Listing11.php(47) l…_9_10_tests::assertformworks(...) … D:\prj\Listing11.php(75) phpunit::run(...) Test case failed: testgothroughform_usetags Error message: expected , actual
testGoThroughForm_UseTags was written to test for the case of user input that contains tags. The test is now telling us that it was expecting to find the text “
Back to Listing 9
September 2005
●
PHP Architect
●
www.phparch.com
on the page, but it found nothing. This is because the user input, “c:\php\cli\php Listing11.php TestCase listing_9_10_tests->testgothroughlink() passed TestCase listing_9_10_tests->testgothroughform() passed TestCase listing_9_10_tests->testgothroughform_usetags() passed
The HTMLDocument Wrapper Object When you call BrowserEET_TestCase::loadDocument() from your test case, it eventually calls Browser::loadDocument(). However, what you get back is not the actual Internet Explorer document object. Instead, you get an instance of the HTMLDocument class, a wrapper for the IE document object. HTMLDocument is closely tied to your test case instance and lets you perform various assertions (such as assertNoPHPScriptError() which we saw earlier). It also provides utility functions such as clickOnElement(), which is used for simplifying the process of clicking on an element.
49
FEATURE
End-to-End Testing
The function that you will use most often is HTMLDocument::getElementById(), a function that looks for an element in the document and returns a reference to it. By default, getElementById() causes the test case to fail if it is unable to locate the specified element. This is useful for detecting missing elements, as well as for detecting bugs in the test script itself. You can disable the existence check if you know that your element might legitimately be missing. You can see sample usage of getElementById() in Listing 11, line 32. If your element lacks an id attribute, you can find it name through a attribute using HTMLDocument::getElementByName(). Using names is not as good as using identifies for two reasons: It is legal to have more than one element with the same name. For example, each option in a radio group has the same name. Internet Explorer’s getElementsByName() will not find all types of elements. getElementByName() assumes that there is exactly one element with the specified name. If it finds more than one, it will complain. To make sure that you are on the correct page, use assertUrl(). For example, if you have a form that dynamically decides the destination of its submission, you would write the following code: $frm = $doc->getElementById(‘myFrm’); $frm->submit(); $doc &= $this->loadDocument(); // make sure the form was submitted the correct page $doc->assertUrl(‘/expected_page.php’);
assertUrl() takes your base URL into account when performing its check. Designing For Testability Many years ago, I worked for a company that wrote a complex client/server application. The client had very nice, rich GUI. The problem was testing it. The QA department got tired of clicking on the same button day in and day out, so they decided to write some automated scripts to do their job. Great idea! Except that the GUI wasn’t written with any sort of automated testing in mind. They were forced to buy a testing tool that cost more money than I made that year. This tool was great at playing back “scripts” that the QA department recorded, right down to clicking on the same exact pixel. But, you see, that was the problem. If the developers decided to move a button 5 pixels to the right, the QA department’s script broke right away. The scripts were so hard to maintain, that they were only updated and run very close to releases. Now, imagine a test script that can find the button regardless of its pixel position. The script would be more durable and easier to maintain, and will therefore be used more often. The programmers can move their
September 2005
●
PHP Architect
●
www.phparch.com
buttons around, QA can get its job done, bugs will be found faster, and in general, everybody is happier. In our end-to-end tests, we have many equivalents to this aforementioned button. Every link, every text element and every image we want to interact with, programmatically, should be tagged in such a way that we could easily find it. The natural way to tag HTML items is by giving them an id attribute. After that, finding them is as simple as calling getElementById(). If you need to tag text within the document, simply surround it by a <span> tag, and give the span an identifier. This is exactly what I did in Listing 10. Listing 13 demonstrates some of the complications of writing a test script for a site that was not designed for end-to-end testability. In this case, the site is http://www.php.net/. Listing 13 includes two tests: one tests the site’s search functionality, and the other tests the site’s ability to generate a printer-friendly version of a manual page. The search box test starts out by populating the search field with a value of “ddebug_backtrace ”. Although the text input element has no identifier, it does have a unique name, so we are able to find it using getElementByName() (line 22). To submit a form, I usually prefer to click on the “submit” button. This most closely simulates what the user will be doing, and will allow any event handle attached to the button to fire. Unfortunately, the submit button for the search form (in this case, it is actually a submit image) is unnamed and has no identifier. Although I had the option of writing a complex script to find this image based on something like the value of its src attribute, I chose to submit the form directly using its submit method. This way, if the graphics department decides to change the image file names, the test doesn’t break. Once the test lands on the target page, we discover the next obstacle: we have no easy way to ensure that we are, in fact, looking at the correct page. It would have been nice if the name of the function the page was displaying was retrievable in some deterministic way. Unfortunately, the designers of this page didn’t Listing 11 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Listing 12 1 // This function scans through the document, looking for 2 // PHP errors and notices. 3 function assertNoPHPScriptError() 4 { 5 $innerHtml = $this->theDocument->body->innerHtml; 6 7 if (// standard PHP warning 8 preg_match(‘/.*:.*in .* on line \d*/i’, 9 $innerHtml, $matches) 10 11 || 12 // PHP warning if xdebug is installed 13 14 preg_match(‘/[^>]*: .*in .* on line \d*/i’, 15 $innerHtml, $matches) 16 ) 17 { 18 echo $innerHtml; 19 $this->parentTestCase->fail(‘PHP Notice Detected: ‘ . $matches[0] . ‘ url=’ . $this->theDocument->url); 20 } 21 } 22
September 2005
●
PHP Architect
●
www.phparch.com
1
51
FEATURE
End-to-End Testng
have my test script in mind when they created it, so I had to resort to the knowledge that, in the current site design, the function name is placed in within a tag. Thus, the test looks for the following text inside the page: debug_backtrace
Again, this test is at the mercy of the graphic designers. If they no longer want to bold the function name, the test breaks. In a site designed for testability, we would add a <span> element around the function name, so that we could retrieve the function name without relying on the surrounding formatting tags: <span id=”idFnName”>debug_backtrace
The second test encounters all of the same obstacles. Most of the code is devoted to figuring out which option to select in the “page type” combo box. To submit the combo box selection, the script locates the proper element through the combo box’s form property (line 68). This is necessary because the form has no identifier and no name. Finally, to validate that we are on the proper page, the script is forced to look
for specific text strings. With minor changes to the web page, these tests would have been easier to write, and they would have been more tolerant of changes to the web page. When writing your own page scripts, keep the tests in the back of your mind, and tag your elements accordingly. Limitations I’ve mentioned multiple times that with the full Internet Explorer under our control, we can do many things that might otherwise seem impossible. However, there are some areas where this approach is still limited. The first thing to consider is interactivity with the JavaScript GUI. If your script uses the JavaScript alert() or confirm() functions, there will be no way to dismiss the resulting dialog box, and the test script will hang. A workaround, which requires replacing alert() and confirm() is beyond the scope of this article. Next, you’ll need to consider how you’ll directly call JavaScript code. I have been unable to directly call JavaScript functions through PHP/COM. This means that JavaScript code on the web page can’t be tested directly. Load Testing is another thing to consider. I’ve previously mentioned that launching more than one brows-
Internet Explorer Quirks Internet Explorer is a dinosaur. It is an old codebase, and has never fully supported W3C standards. When dealing with Internet Explorer, it is important to note a few idiosyncrasies. Automatic id attributes The name attribute generates an automatic id attribute. In this nonstandard behavior, a tag that has a name attribute but no id attribute will behave as though it has an id with the same value as the name. This means that the following two code segments are equivalent: (A) (B)
This could lead to a problem like the one demonstrated by the code below. In Firefox, the code outputs “red” as expected, but in IE you will see the value “blue”, as IE is incorrectly looking at input tag and not the span tag: <span style=”color:red” id=”label”>foo <script>
September 2005
●
PHP Architect
●
www.phparch.com
alert(document.getElementById(‘label’).style.color)
innerHTML Does Not Return your HTML The innerHTML property does not return the actual source HTML. Instead, it returns HTML that has been preprocessed by Internet Explorer. IE will convert all tag names to uppercase, remove quotes from attributes, change white-space and convert tags such as
to
. <span id=”test”> Hello
<script> // displays: Hello
alert(document.getElementById(‘test’).innerHTML);
Nonstandard Extensions Internet Explorer supports some non-standard constructs that can be helpful during testing. One of the more useful is the innerText property, which is similar to innerHTML but does not return tags. Others that are worth mentioning are document.all and outerHTML.
52
FEATURE
End-to-End Testing
er from our PHP script could cause problems. With just the one browser, a single test script can’t be used to perform load testing on a site. Of course, you can always spawn multiple test scripts at the same time, but this is an external solution. The last limitation I’ll cover is the fact that you’ll have no access to raw HTML. When you use the innerHtml property, you get a version of the HTML that was preprocessed by Internet Explorer. This is no longer the HTML that you wrote, but rather a version that was normalized for/by the IE rendering engine. For more information, see the Internet Explorer Quirks sidebar. This limits your ability to write tests that require raw access to the source (e.g. validating that you are generating valid XML). Possible Enhancements BrowserEET_TestCase and its supporting classes, as presented here, can serve as a starting point for a robust end-to-end testing framework. Some areas where these classes can be extended are: Support for sessions via URL rewriting. When the session URL rewriting engine is active, all URLs generated by PHP have a PHPSESSIONID parameter appended to them, by default. The framework should ideally detect the session identifier and automatically append it to
September 2005
●
PHP Architect
●
www.phparch.com
loaded URLs, as necessary. Support for frames. Currently, the framework does not directly support sites with frames. Most methods should be expanded to handle frame sets and internal frames. More helper functions. An almost endless number of utility functions can be added. For example, functions for manipulating standard form controls would be very helpful. Wrapping Up In this article, I presented the merits of end-to-end testing and introduced a framework for writing end-to-end test scripts in PHP. These simple to write test scripts are a powerful way to test any web site (even sites not written in PHP). End-to-end tests can help you to greatly improve the reliability of your product. I hope you use them well! About the Author
?>
After over a decade of nothing-but-C++, Oz was luckily introduced to the world of PHP, which his team now uses to develop web-enabled applications. Oz lives and works in Toronto. He can be contacted at http://www.wndtabs.com.
To Discuss this article: http://forums.phparch.com/251
53
TEST PATTERN
State of Confusion by Marcus Baker
Bug-free code is code you can understand completely. Unfortunately, it doesn’t take much to cause us uncertainty. Even a small amount of complexity will place demands on our memory, and our memory is extremely limited. If we tax our brains, we will get bugs. We are only human.
W
e can keep approximately six things in our short term memory at once. This rule of thumb is pretty well known. I know it, and I am sure you do to. So what was I thinking when I wrote the code in Listing 1? As a newly created Person object gets passed around, its methods will get called and it will change its internal variables. I am not so worried about the properties such as $data , which contains the simple database information—the person’s name and the like. I am much more concerned about the internal state of the object. For example, the $is_external flag actually means that the person is not part of the in-house database and so cannot be modified. This is a variable that changes the behavior of the object. If I am to understand the code that I write, I have to keep track of those state changes in my head. If I fail to do that, I will get confused and start to produce subtle bugs when methods don’t act quite the way I expect them to. Here is a nasty example... $person = new Person($connection); $person->loadById(33); $person->makeExternal();
Later on, I may do this... if ($person->begin()) {
September 2005
●
PHP Architect
●
www.phparch.com
REQUIREMENTS PHP
5
CODE DIRECTORY
test
$person->setLogin(new Login()); if (! $person->isExternal()) { $person->commit(); } }
These two snippets don’t match up, because in the first snippet, I declared the person external, so it cannot be saved. Or can I? Is allowing a log-in capability a special case? How do I know it was loaded successfully, and when I loaded number 33, how did I know that number 33 is supposed to be external? How do I know that this is the only call to begin a transaction? What if I start two? If I want to play safe, I will have to write a whole bunch of code to double check this. That creates work not just for me, but for others too, so I would prefer to get it right by inspection. You can see part of the execution path in Listing 2. Painful isn’t it? Sadly, this class has so many member variables that there is no way I can keep track of all of them, in my head. I just don’t have the mental capacity. I have written a bug factory. Excessive member variables are not the only source of error. A method longer than five lines or so is likely to be just as confusing. The difference is that a method is at least localized. I can test it thoroughly and as long as the method name is well chosen, it won’t cause any
54
TEST PATTERN
State of Confusion
external damage to other code. When the bug occurs, it can be easily traced back to the substandard method. Not that you should have methods longer than about five lines, of course. In comparison, state changes are especially insidious, because each state change causes the object to behave differently. This can easily make the usage of the object flawed, even if the code that makes up the class itself is correct. It spreads confusion. “It must...” My example is pretty blatant. Because of all the other mental balls that you juggle at any one time, it will probably take far less than nine variables to cause you to start dropping them. Now, I cannot determine the correct number of member variables for a class. It
Listing 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Listing 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
September 2005
●
PHP Architect
●
www.phparch.com
depends on the complexity of the situation. Luckily, there are clues when things start to slide. It is very obvious when we are in control. We can actually “see” the program flow as of we are looking at a picture. We take it in at a glance, rather than having to step through each element item by item. Having to single step is a warning sign that we are approaching our mental limits. Once we can no longer single step, we start to grasp at constants to reduce the complexity. We make statements such as “because it has X it must be doing Y”. As soon as we start resorting to tortuous deductions to get a handle on our code, we have definitely gone off of the rails. Once I hear myself utter “must,” I get nervous and feel the need to refactor. How do we refactor? Simply splitting the class into smaller objects is good, but is not a panacea. The top level class has fewer variables, namely the subdivided objects, but these are more complicated in themselves. The state hasn’t gone away. We will reduce interactions, though, so a finer-grained approach is part of the solution. To divide more incisively, however, we must include some other tricks. Die by the Sword Take a look at Listing 3. This type of code is so familiar in the PHP world that it probably doesn’t even seem interesting. The Socket object actually has three states: unopened, open and closed. Of course, if you write to the socket when it has been closed we would expect an error or an exception. This means that both the Socket class and the client code will be riddled with if or try statements to manage the possible error conditions every time data is sent or read. Listing 4 is a little stricter. We have taken advantage of PHP 5 exceptions to avoid carrying errors. By not allowing the object to be created unless the socket is connected, we remove the unopened state altogether. The new operator simply will not return an object on failure. This is known as the Samurai Principle—“return victorious or don’t return at all!” We remove the closed state by taking advantage of destructors. The client code must actually unset() the socket variable or let it drop out of scope. Of course, the socket could be closed at the other end, so try blocks are still forced, but external events are part and parcel of network programming. At least this way we haven’t made things harder by having to avoid transmission after we called close(), ourselves. By matching the object lifecycle to the problem domain, we have avoided self-inflicted states. We have to be a little careful about cloning, here. If we have two copies of the object, then destroying one will close the connection for the other. Throwing an exception on clone avoids that issue. At this point, we have been so successful that the
55
TEST PATTERN
State of Confusion
object has no visible state to the outside world at all. It’s just an open socket, end of story. Push Mechanics Down Stateless classes are called “service classes” and are the easiest classes to understand. No matter how many times we call read(), the Socket object will always do exactly the same thing. It has no knowledge; it just does mechanical work when requested. A characteristic of a good service classes is that they must be told everything. Constructor parameters are constant, and everything else must be passed into the actual method call. This has the effect of pushing knowledge out into the client code that uses it. Our Socket class forces the client to know which host to connect to, for example. That may sound bad, but it isn’t. Service classes work well when there is a technical operation. By moving all decisions and set up into the client code, we keep that code clear of mechanical details. The client states what should happen and the preconditions, but nothing about the how. The complicated service class has the minimum of dependencies on the outside world and so becomes very predictable. This makes testing easier for the mechanics, and makes the logic of the client much more declarative. Mechanics down and policy up is a good rule of thumb. This is actually a very subtle example of “cohesion,” or one responsibility per class. The choice of host and the actions to carry out on failure will all be in the same place—the client of Socket—and laid out in front of our eyes. That means that all of the state interactions are visible and separated from the behaviour code. We no longer have to predict behaviour while keeping hidden state in our head.
Listing 3 1
Listing 4
Transactions OK, back to our Person class. An obvious piece of lifecycle is the commit() and rollback() behaviours. This is a policy decision for sure, but one that is fairly high up in the application. Instead of passing in a standard database connection object, let’s pass in a service that handles transaction behaviour for us. We’ll call it a Transaction (Listing 5). Unlike the Socket example, we don’t place the commit() method in a destructor. This is because the commit() method is likely to throw an exception that cannot easily be dealt with while the script is already shutting down. Clean up is a delicate process and we could toss a real wrench in the works for users of our class by throwing an exception right in the middle. The transaction approach has several advantages. Firstly, we can map the object lifecycle of the Transaction to the begin() behaviour, and remove that method. Secondly, we actually push the decision to commit upwards from this class. It’s likely that there will
September 2005
●
PHP Architect
●
www.phparch.com
1
56
TEST PATTERN
State of Confusion
Listing 5 1
Listing 6 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
●
PHP Architect
Initialization is Construction The next obvious cut is to push down the mechanics of actually writing to the database. The different ways of managing this split is a book in itself, but I’m going to go with the most naïve, for simplicity. We will just create an object internally for delegation that will handle all of the database chit chat. This object is usually called the DataAccessor or DataAccessObject (DAO). The outline of this is in Listing 6, as PersonAccessor. With the mechanics delegated away, our Person should become a little simpler. It’s really more of a data holder that can save itself, the only set of mechanics left, is the loadById() method. This is a pretty lousy method. The original idea was that if you wanted to create a new Person you would do it like this: $person = new Person($connection);
function __construct($snapshot) { sort($snapshot); $this->snapshot = $snapshot; }
September 2005
be operations with several people and other objects and so it makes sense for the transaction to be shared across the application. After all, it will be the application that has to sort out the mess if the commit() fails, so let’s have everything in one place. From the point of view of the Person class, we don’t care about any of this. We just execute queries and assume success. Any failures, and the exception will be thrown for us. We never worry about failure and error states, although our Person object still has the power to issue a rollback() if needed. This trick of turning a continuous connection into a single usage is also known as the UnitOfWork pattern (from Patterns of Enterprise Application Architecture by Martin Fowler). It has particular applicability to the world of PHP, because of the constant hit and run nature of a web application built from small scripts. The UnitOfWork is also an excellent place to keep track of uncommitted objects. We won’t cover that here, but suffice to say the last listing in this article still has plenty of room for refactoring.
●
www.phparch.com
If you wanted to load a specific person, you would do it like this: $person = new Person($connection); $person->loadById(33);
While execution is between these two lines of code, the object is in a state it shouldn’t be in. Any changes that are accidentally made will be mangled when loadById() is called, possibly causing a complete mess. Even more unpredictable things could happen if we accidentally called the loader on an object we were already working with. This is another example of having to manually keep track of state to avoid disaster. The C++ community has a phrase: “initialization is construction.” It basically says that once you create an object, it should be ready to use in one go, with no fur-
57
TEST PATTERN
State of Confusion
ther steps to take. We can wrap our multi-step process by having the Person object created by another function, which I’ll call findById() in PersonFinder (Listing 7). This method handles the database fetch, as well. This means that the Person only needs to know about the Transaction when it’s time to save itself. This final escape clause means that we don’t have to keep track of any kind of database connection at all. That’s another member variable gone. I feel the weight being lifted from my shoulders as we progress. Type Splitting There is one last piece of nastiness that has been bugging me for some time. Remember the $is_external variable? Certain Persons are read only, because they come from another database system. If set, this variable prevents the save() method from working. This is a flag, and flags are evil. The problem is that we are effectively dealing with two different types. By flipping that flag, we convert the object from one type to another. Now, PHP already has a type system, called a class. That flag splits a single class into two types and externally we have to keep track of which one it’s pretending to be. Split types generate confusion for no justifiable reason and should be avoided. If you see flags like this, then simply divide the code up into two separate classes, say Person and ExternalPerson. Not only is this less confusing to use, but the internal code will be much cleaner too. The ExternalPerson won’t need a PersonAccessor, for example. For brevity, I will skip this as a code listing, and simply explain that the PersonFinder should be able to deduce this from the data and instantiate the correct class. A possible example of this is included in the Listing 7 code. Finally... The Person class is reborn in Listing 8. Let’s try using it... $finder = new PersonFinder(); $person = $finder->findById($transaction, 33);
And later on... $person->setLogin(new Login()); $person->save($transaction); $transaction->commit();
Any exceptions that are thrown can safely be passed upward to the controlling script. This is because any errors are genuine, caused by database failure or some other terminal condition. As you follow the code in the methods that are called in Listing 8, you can see that the statements are emphatic. You don’t have to memorize anything because there is just nothing to remember. You can
September 2005
●
PHP Architect
●
www.phparch.com
take the whole thing in, at a glance. The only state change is when the two calls to save() are made, the second will have no effect, because the accessor will not generate a query. You could argue that this isn’t a state change anyway, as the behaviour is still the same—leaving the database in sync with the Person object. All other states result in an exception and a rollback, leaving the top level script with nothing to do except display the error page. This new code is brain sized with room to spare. It’s not just small; it’s trivially small. As programmers, we need that spare capacity.
About the Author
?>
Marcus Baker works at Wordtracker (www.wordtracker.com) as Head of Technical, where his responsibilities include the development of applications for mining Internet search engine data. His previous work includes telephony and robotics. Marcus is the lead developer of the SimpleTest project, which is available on Sourceforge. He's also a big fan of eXtreme programming, which he has been practising for about two years.
To Discuss this article:
http://forums.phparch.com/253 Listing 7 1
Listing 8 1
58
PRODUCT REVIEW
Discussion Forums Made Easy 2.7.1 by Peter B. MacIntyre
T
his month’s product review covers the popular product known as FUDforum. It is an open source (PHP developed) forum management tool. This product can lend itself to managing group discussions on any topic. We actually use this product here at the magazine to manage our discussions on the articles that are published, as well as other magazineand conference- related issues. The developers of FUDforum list the following as their product’s main features: FUDforum is a highly customizable forum package, with a large feature set. The easy to use administration control panel allows the administrator to easily configure and control the many features available. The forum also includes an interactive help package, which helps to familiarize users with the full potential of FUDforum.
September 2005
●
PHP Architect
●
FUDforum is shown in action in the figure above, taken from php|architect’s discussion site: www.phparch.com/discuss
PRODUCT INFORMATION PHP
4+, 5+ (MySQL 3.23+)
OS
Any
Product Version
2.7.1
Price
FREE
Web Address
http://fudforum.org/forum/
www.phparch.com
59
PRODUCT REVIEW
FUDforum 2.7.1
Here are some of the core features offered by FUDforum: • i18n support, with a number of languages distributed with the forum • Full template support with a FAST template compiler • Powerful group management system, which allows fine-grained control over forum’s resources • Private Messaging system, with post tracking & buddy lists • Built-in spell checker (using the pspell library) • Ability to use FUDcode, HTML or Plain Text inside posts, configurable by the admin • Administrator-definable ranks for the users, based on number of posts as well as the ability to assign an unlimited number of custom tags to individual users • 3 types of user avatars • Ignore list allows forum members to ignore posts made by other users
• Threaded View (by default standard flat view is used) • Customizable “Find and Replace” system, which fully supports both perl regular expressions (ppreg) and even str_replace() for extremely fast simple replaces • NNTP & Mailing List integration, allowing FUDforum to be used to archive newsgroups and mailing lists, as well as allow forum members to post messages to newsgroups and mailing lists via the forum Laying the Ground Work With the above list of features (which just scratches the surface), it was rather hard to know how to approach doing a review on this product without leaving any of the good stuff out. Hopefully I have done FUDforum a service in this review. I’ve also used this product myself for a soccer referees’ discussion site, and I will be using that site in this review as one of the example sites. If you are interested in soccer
refereeing (as I am), then you can visit the site at http://refereeforum.paladin-bs.com .
Once you obtain the software from the project’s web site, simply extract the files into the web server area and run the install.php script. The installation process will begin, with the first step, as shown in Figure 1. Keep in mind that FUDforum can keep all of its information in a MySQL database structure so that the related information can be tracked and displayed more easily. Having said this, it is a requirement of the install process of FUDforum that a database pre-exist for it to be used. All of the tables that are required by FUDforum will be generated automatically, however the database name itself will have to already exist for the install to work. This is a very minor issue. Figure 2 shows another one of the strengths of this product, it is not just an English based interface. There are over 20 different languages that can be used—now that’s flexibility! After completing a few basic forum setting, and completing a successful installation, you will be asked to sign in with the
Figure 1
September 2005
●
PHP Architect
●
www.phparch.com
60
PRODUCT REVIEW
FUDforum 2.7.1
Figure 2
Figure 3
administration information that you provided during the install. Administering the Site Once signed in you will be taken to the main administration page where you can continue to set up the details of your site. Figure 3 shows a listing of some of the administration menu options. As is quite evident here, you have
a lot of control on the system and a lot of flexibility. For example, if you click on the “Category and Forum Management” menu item you will be taken to the admin area where you can set up you individual discussion areas, known as categories, and sub-discussion areas known as forums. This is shown in Figure 4. Figure 5 shows the FUDforum that is active on my soccer referee’s
Figure 4
September 2005
●
PHP Architect
●
www.phparch.com
61
PRODUCT REVIEW
FUDforum 2.7.1
web site,5 it shows me as being Figure
allow you to manage your site. You
of FUDforum’s workings you can
logged in, and shows some of the discussion areas that we have created. This site is a “closed” site in that it only allows known users to register on the forums. We use this feature to screen out any potential slander that may be brought against a local referee. (And what referee is not prone to some kind of abuse, right?) At any rate, the administrator of the site controls the feature of open or closed memberships. There are many other control features that
can ban problematic posters if desired. You can have e-mail announcements sent when a new message is posted to a given forum. You can have personal messages between members, and so on. This product has so many options that it would do you well to install it on a testing server and allow only a select few to use it at the outset. You can then turn on and off certain features to see what they really do for you. Finally, as you and your test group become familiar with all
release it to your whole group of users. Summary Once you get familiar with the concepts and terminology of how to run the FUDforum software, I am sure you will be very happy with its performance. After you set things up, the software really just runs itself. There is even a way to extend the forums’ look and feel by surrounding it with banners. This can be seen on php|architect’s forum site, with a banner and a left-hand side bar. FUDforum is a very stable and mature PHP product; even the help documentation is well done and quite complete, which is a rare thing for open source. I give this product 4 out of 5 stars.
Dynamic Web Pages www.dynamicwebpages.de sex could not be better | dynamic web pages - german php.node
news . scripts . tutorials . downloads . books . installation hints
September 2005
●
PHP Architect
●
www.phparch.com
About the Author
?>
Peter MacIntyre lives and works in Prince Edward Island, Canada. He has been and editor with php|architect since September 2003. Peter’s web site is at http://paladin-bs.com
62
SECURITY CORNER
PHP Security Audits by Chris Shiflett
Welcome to another edition of Security Corner. This month’s topic is the art of the PHP security audit, a procedure that can be integrated into your development process to vastly improve the security of your PHP applications. Although auditing code is a highly specialized skill, peer reviews are one of the most useful and underutilized development techniques.
PHP security audits are one of the professional services that I provide through Brain Bulb, and I have accumulated quite a few tips and tricks that I want to share. Although professional security audits tend to be quite expensive, peer reviews can offer a great deal of value at a decreased cost. With a moderate understanding of how to audit PHP code, you can be a valuable asset to any PHP development team.
B
words, you need to set the bar. I recommend starting with a minimum goal. It’s very difficult to gauge the effort required to audit an application, and you should focus on the most important things first. Depending on your employment situation, you might have new responsibilities materializing as you go, new assignments to steal your focus, or you might simply run out of time. Rather than risk focusing on the details and missing something major and obvious, it’s best to look at the big picture first. You can always dive into the details later. The bar that I use is that every PHP application should at least filter input and escape output (“FIEO”). This involves more than just performing these steps—it also suggests that the application should make sure that tainted data cannot possibly be mistaken for filtered data, the filtering process cannot be avoided by a clever attacker, and the like. For example, while you’re examining the source code, you might encounter code like the following:
efore you get started with any task, it’s always best to know what exactly you’re doing. Before you can effectively audit a PHP application, you need to clearly define your task and its associated goals. What are you doing, and why are you doing it? What is a PHP security audit? An audit is an examination, so a PHP security audit is primarily an examination of a PHP application’s source code. In other words, it’s a code review with a narrow focus: security. There are a few abstract points of interest, including the software design and the PHP configuration. In order for an audit to be as valuable as possible, it is important that nothing be off-limits. The idea that “a chain is only as strong as its weakest link” certainly applies, so you want to have access to everything, otherwise you might not find the “weakest link” in the application. Setting the Bar There are a few steps to take before you actually start examining the source code. One of the first steps is to determine how much security is required. In other
September 2005
●
PHP Architect
●
www.phparch.com
63
SECURITY CORNER
PHP Security Audits
It’s hard to tell from this one line whether $action and $desc have been properly filtered and escaped. This is a security risk. In this particular case, it might be possible to restrict $action to a set of valid values, like I’ve done in Listing 1. Of course, this filtering is unnecessary if $action is set in the code and does not come from a remote source. It’s still better to adhere to a strict naming convention, otherwise it’s difficult to tell whether $action is filtered or tainted:
Remember, you must filter input, but data that is not input does not have to be filtered. If this data is used as output, however, it still needs to be escaped. Because $action is being used as the value of an argument in the query string, the escaping that is required is URL encoding:
Note: this illustrates a point of confusion for many developers, especially those new to security concerns. When identifying output, anything sent to the client is output, even URLs or form data that is meant to be sent back to the server. In this case, $action is being sent to the client. Do not confuse this with $_GET[‘action’], the variable you reference when the user clicks on this link. Because $desc is sent to the client, it must also be escaped. In this case, the proper escaping is htmlentities() . Here is an example that assumes $clean[‘desc’] is the filtered description:
Of course, when you’re just auditing code, your task is to make sure that these required steps have been taken. Identifying failures is enough, although providing an exploit can often help to clarify a vulnerability. Analyzing the Design Another step to take before you take a detailed look at the code is to analyze the design. I always begin this process by having the design explained to me, preferably by the developers. No one knows an application as
September 2005
●
PHP Architect
●
www.phparch.com
well as the developers, and implementation often strays slightly from the documentation that might be available, so the developers are the only reliable source of information in this regard. A poor or unnecessarily complex design is a security risk. It can be the most impressive design you have ever seen, but if the developers can’t properly explain it due to its complexity, then it represents a possible security hole. Complexity breeds mistakes, and mistakes frequently yield security vulnerabilities. Another indication of a design problem is when tracking data is difficult. Can you easily track data from the point where it enters the system to the point where it Listing 1 1
SECURITY CORNER
PHP Security Audits
exits, including transformations? If not, then it’s likely that the developers can’t either, and this is a security risk. As noted in the previous section, it’s also important that distinguishing between tainted and filtered data is made easy. If this is difficult, then developers are more likely to mistake tainted data for filtered data, and will almost certainly write vulnerable code. Last, security must be part of the design. A design with no mechanisms to help promote security is the biggest mistake you can identify. Security-conscious developers cannot compensate for a lack of security in the design, and many PHP applications suffer from this. Without a secure design, developers are destined to be perpetually patching security vulnerabilities. Analyzing the Configuration The last step to take before examining the source code is to analyze the configuration. PHP’s configuration is mostly dictated by php.ini, but don’t forget that it can also be modified by things like httpd.conf, .htaccess files, and ini_set() calls. Things to avoid include: • register_globals = On • allow_url_fopen = On • display_errors = On • magic_quotes_gpc = On In general, if the security of the application depends upon the configuration, this is a risk that needs to be mitigated. Examining the Source Now you’re ready to actually start examining the source, but where do you start? This is where it is important to have already set the bar. To check whether an application adheres to FIEO, there are two steps to take: Identify input, and trace it forward. Identify output, and trace it backward. These steps are a bit redundant, but they can provide you with two different perspectives, and taking both steps can help eliminate failures. If you’re auditing an application, you’re being trusted to identify all major vulnerabilities. Redundancy is good. There are several ways to identify input, and I usually use grep or some custom utilities to help me search. HTML forms are the primary way that an application receives input from the user, and there are several strings you can search for to help you find them: • form • input • radio • select • checkbox
September 2005
●
PHP Architect
●
www.phparch.com
• $_GET • $_POST • $_REQUEST Databases are probably the second most common source of input, and SELECT statements are worth inspecting. Remember, though, that SQL is case insensitive. Discovering a developer’s habits can help, because developers tend to be consistent. However, you can’t guarantee this, so it’s best to search for both uppercase and lowercase versions: • SELECT • select If you want to be extremely thorough, you can search for all permutations of uppercase and lowercase, but I don’t think this is very useful. Any developer who uses SeLeCt needs to find a new career anyway. HTTP headers can be accessed directly in PHP, so this is something else worth searching for. Some helpful strings to search for include: • $_COOKIE • $_SERVER Remember, locate sources of input and trace the code forward. The code in Listing 2, for example, presents a case where it’s difficult to recognize that $url is tainted by examining the last line out of context, but if you follow the data in $_GET[‘action’] through the source code, this becomes obvious. In the next step, where you identify output and trace it backward, you should be able to discover the same Listing 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Listing 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14
SE CURIT Y CO RN ER
PHP Security Audits
vulnerability. As with input, there are several ways to identify output. The major recipient of output from a PHP application is the client, and there are several ways to send output to it. The following strings are useful searches: • echo • print •
Chris Shiflett is an internationally recognized expert in the field of PHP security and the founder and President of Brain Bulb, a PHP consultancy that offers a variety of services to clients around the world. Chris is a leader in the PHP industry, and his involvement includes being the founder of the PHP Security Consortium, the founder of PHPCommunity.org, a member of the Zend PHP Advisory Board, and an author of the Zend PHP Certification. A prolific writer, Chris has regular columns in both PHP Magazine and php|architect. He is also the author of the HTTP Developer's Handbook (Sams) as well as the highly anticipated PHP Security (O'Reilly). You can contact him at
[email protected] or visit his web site at http://shiflett.org/.
To Discuss this article:
http://forums.phparch.com/254
66
You’ll never know what we’ll come up with next For existing subscribers
NEW
Upgrade to the Print edition and save!
LOWER PRICE! Login to your account for more details.
php|architect
Visit: http://www.phparch.com/print for more information or to subscribe online.
The Magazine For PHP Professionals
Address: _________________________________________ City: _____________________________________________ State/Province: ____________________________________
E!
WE
RP
RIC
W
*US Pricing is approximate and for illustration purposes only.
NE
Name: ____________________________________________
Your charge will appear under the name "Marco Tabini & Associates, Inc." Please allow up to 4 to 6 weeks for your subscription to be established and your first issue to be mailed to you.
Choose a Subscription type:
Canada/USA International Air Combo edition add-on (print + PDF edition)
LO
php|architect Subscription Dept. P.O. Box 54526 1771 Avenue Road Toronto, ON M5M 4N5 Canada
$ 77.99 CAD $105.19 CAD $ 14.00 CAD
($59.99 US*) ($80.89 US*) ($10.00 US)
ZIP/Postal Code: ___________________________________ Country: ___________________________________________ Payment type: VISA Mastercard
American Express
Credit Card Number:________________________________ Expiration Date: _____________________________________
Signature:
Date:
*By signing this order form, you agree that we will charge your account in Canadian dollars for the “CAD” amounts indicated above. Because of fluctuations in the exchange rates, the actual amount charged in your currency on your credit card statement may vary slightly.
E-mail address: ______________________________________ Phone Number: ____________________________________
To subscribe via snail mail - please detach/copy this form, fill it out and mail to the address above or fax to +1-416-630-5057 67
exit(0);
Atomic Orange by Marco Tabini
F
or a business—any business— growing is good, and painful (hence, I suppose, the proverbial phrase “pain is so close to pleasure”). Sometimes, once in a blue moon, it’s also funny. As our book business keeps becoming more and more successful, I keep having to find better ways to print our books. The economy of printing is a funky one—I’ve always thought that accountants who used to work in the printing industry must have been responsible for the foundation of the Hollywood economy, where a movie can make half a billion dollars without ever turning a profit— and difficult to navigate for the beginner. Suffice it to say that the vast majority of the costs connected with printing a book is not related to the actual printing, but rather to the set up—developing the film, etching the plates, and so on. Thus, printing, say, one thousand copies of a book can cost only marginally less (in relative terms) than printing 10,000 (the problem being, of course, where you chuck the other 9,000 copies you don’t actually need). Besides making for interesting coffee-time talk, this also means
September 2005
●
PHP Architect
●
that the cost of printing the same book varies wildly between printers, depending on how quickly they think they can set up a title, and how much they charge for their set up work. As a result, the moneyconscious publisher (or, as other people prefer to refer to me, the tight tyrant) finds himself having to shop around for the best price on a book-by-book basis. A couple of weeks ago, as we were about to run the first print of Jason Sweat’s php|architect’s Guide to PHP Design Patterns, I happened to drop a request for quotation with a printer we had never used, but whose pricing structure seemed very promising. Mindful of how much I dislike clients who send me requests for proposals without enough information to provide a meaningful price, I made sure to include as much detail as possible with my RFQ—trim size, page count, run size, and so on. Unlike the software world, where pretty much every project is completely different from the next, there is only so much that can be done with a book; therefore, I wasn’t expecting much in the way of problems. Imagine my surprise when I received a quote for something
www.phparch.com
that, had the title of our book not been on it, I would have simply thought had been faxed to the wrong speed-dial. Wrong trim size, spiral binding (uh? Did I order notebooks?) and quantities high enough to teach the savage people of the Deep Amazonian Forest, as yet untouched by western civilization, the ins and outs of Iterator pattern. To me, that’s like asking how much two pounds of oranges cost to your local grocer only to be told the price of three tonnes of fissionable plutonium. So, the morning juice is out of the question, but the afternoon build-your-own-nuke fest is on. Needless to say, the printer didn’t even get a call back. Everybody makes mistakes… just as long as they don’t make them with my stuff. In business, like in so many other aspects of life, first impressions are very important, and my first impression of this company was that I’d be shipped ten thousand copies of the latest Martha Stewart Living, which I have a feeling most of our readers wouldn’t have been interested in.
php|a
68
