php|architect (December 2002)

DECEMBER 2002 - Volume I - Issue 1 php|architect The Magazine For PHP Professionals Zeev Suraski - Brad Young PERFORMA...

Author: Larry Ullman

27 downloads 750 Views 1MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

DECEMBER 2002 - Volume I - Issue 1

php|architect The Magazine For PHP Professionals

Zeev Suraski - Brad Young PERFORMANCE MANAGEMENT OPPORTUNITIES

This copy is registered to: Liwei Cui [email protected]

Comparing PHP With the Industry at Large

Reviewed for you: Zend Performance Suite Davor's PHP Editor

Plus:

SNMP Management With PHP PHP nCurses Shell Programming Writing a Webmail System Distill PDFs For Free With PHP Create a DB Abstraction Layer

SUBSCRIBE AND

WIN!

In Colaboration with

The designers of PHP offer you the full spectrum of PHP solutions

Serve More. With Less. Zend Performance Suite Reliable Performance Management for PHP

Visit www.zend.com for evaluation version and ROI calculator

Technologies Ltd.

TABLE OF CONTENTS

php|architect Departments 4

| Editorial rants

Features 9

|

Performance Management Opportunities

INDEX

by Zeev Suraski and Brad Young

5

| New Stuff

14

|

Curses! Writing Cool Shell Applications With PHP

6

by Marco Tabini

| PHP-Win A Look at Davor’s

25

|

PHP Editor

SNMP Management With PHP by Harrie Hazewinkel

61

| Product Review Zend Performance Suite

34

|

Database Abstraction Layers

Version 2.5

65

| Tips & Tricks

A Concrete Look at by Maxim Maletsky

41

|

Creating A Visual Catalogue System with PHP and IE

by John W. Holmes

by Davor Pleskina

70

| For Your Reading

49

|

Writing a Web-based

Pleasure

PDF Converter

Book Reviews by the

by Marco Tabini

php|a team

72

| Exit(0); A War With No Enemies

December 2002 · PHP Architect · www.phparch.com

55

|

Directing Mail with qmail’s Maildirs by Jonathan Sharp

3

EDITORIAL RANTS

EDITORIAL

H

ere we are. After months of preparations—and a few sleepless nights—you hold in your hands the very first copy of php|architect! While most editors will spend their first editorial explaining at length the arduous path the brought the fruit of their labor to life, I feel I have a higher goal to aspire to. Getting php|a off the ground has introduced us to some genuinely new experiences—nothing beats being awake at two o’clock in the morning (when you couldn’t read the lettering off a truck if it hit you) trying to pick a highly legible font that will work both on the screen and in print. Still, I don’t think you want to hear about that. Let me, instead, give you an idea of how this issue is organized, so that you can take advantage of it to its fullest potential. We chose PDF and electronic distribution as a move that, we thought, would drive down our costs and, in turn, make it affordable for anybody in the world to subscribe or buy a single issue of php|a. Still, we recognize that there are some unique challenges related to using this distribution strategy, and we have taken a few steps that (we hope) will make it easier for everyone to enjoy our magazine. The PDF file that you have received has been designed both for on-screen viewing and for printing on a regular inkjet or laser printer. This way, you can look the contents over directly on your computer, and you can print whatever interests you—or the whole thing, if you like the feel of paper in your hands. In fact, although the PDF file itself

is encrypted and you should not redistribute it electronically, you can print as many copies of it as you like. If you decide that on-screen viewing is your bag, then here are a couple of pointers. Most of our articles are hot-linked to the Internet—whether it’s an e-mail address to contact an author or the URL of a website from which you can download some great information. Links are an opportunity for us to offer you something that a print magazine will never be able to—a direct connection to the world. Naturally, all our advertisements are also linked—so, show us your support and do visit our advertisers’ websites. They all offer great products. No, really, they do! To make the navigation easier, you will find a full table of contents, which can be accessed from within Acrobat’s bookmarks tab. Notwithstanding all these steps we’ve undertaken, PDF remains a “new” way of publishing a magazine, and we can only hope to get better with time. Your feedback in this process is crucial—we want to hear what we’ve done right, but, especially, we want to hear what we’ve done wrong, so that we can fix it. Drop us an e-mail at [email protected] and let us know what you like and don’t like, what topics you want to see discussed in future issues and any comments you may have about this month’s articles. Happy reading!


php|architect Volume I - Issue 1 December, 2002 Publisher Marco Tabini

Editors Arbi Arzoumani Marco Tabini

Graphics & Layout Arbi Arzoumani

Administration Emanuela Corso

Authors Arbi Arzoumani, Harrie Hazelwinkel, John W. Holmes, Maxim Maletsky, Davor Pleskina, Jonathan Sharp, Zeev Suraski, Marco Tabini, Brad Young

php|architect (ISSN 1705-1142) is published twelve times a year by Marco Tabini & Associates, Inc., P.O. Box. 3342, Markham, ON L3R 6G6, Canada. Although all possible care has been placed in assuring the accuracy of the contents of this magazine, including all associated source code, listings and figures, the publisher assumes no responsibilities with regards of use of the information contained herein or in all associated material.

Contact Information: General mailbox: Editorial: Subscriptions: Sales & advertising: Technical support:

[email protected] [email protected] [email protected] [email protected] [email protected]

Copyright © 2002 Marco Tabini & Associates, Inc. — All Rights Reserved

4

NEW STUFF

NEW STUFF

Open-source Gets a Rap-around As open-source software becomes more and more accepted as a viable alternative to its commercial counterpart in IT industry, it’s becoming painfully obvious that it’s necessary to build an infrastructure to facilitate the exchange of information between developers and customers. Thanks to its open nature, OSS lends itself well to customization— something that is normally unthinkable with commercial products. Since there is no licensing cost, companies find it easy to use opensource projects as the basis for larger initiatives that have very specific requirements. This results in ad-hoc solutions that are based on solid, well-tested code and can be deployed in a short time. Open-source developers are in the unique position of possessing the knowledge needed to successfully market customization and special features to their software

You’ve got a great software product and committed users, but you need additional funding to develop the features they require. packages. Unfortunately, they often suffer from a lack of exposure that would allow them to defray development costs of new features by offering such features to multiple customers at the same time. Marcheka Software Ltd. aims at filling the void between OSS vendors and clients through their Rap-X offering, which can be found at http://www.rap-x.com. Labeled as a “collaborative sales & support platform”, Rap-X works by allowing prospective clients to make requests for new features to be added to software packages in a collaborative way, so that multiple requests for similar changes can be merged into a single one and the development costs can be amortized among several players. With this approach, everyone ends up a winner: developers get the exposure they need and can open


new markets for their products while still benefiting the public at large by keeping their products free. Customers, on the other hand, benefit from the fact that they can enjoy customized software at a fraction of the costs normally associated with it. Rap-x, which already counts five different opensource projects among its clients, is offered both as an ASP (as in Application Solutions Provider, not Active Server Pages!) product and through a more traditional licensing model. You can find its website here. PHP 4.3 Inches Nearer With the release of RC 2 at the end of November , the new PHP version 4.3 is one step from its final release. The new version of our beloved language includes several enhancements and performance optimizations compared to its predecessor. Our favorite one is the ability to fetch SSL-protected pages directly through the fopen() URL wrappers—a major step forward, in our opinion, toward making PHP ever more appealing for commercial applications. You can download PHP 4.3 RC2 from the PHP Group’s main page. Sessions, Anyone? Farpointer Technologies has released a new session management tool for Apache called iConductor. Yes, we know, PHP already supports session management. iConductor, however, is interesting because it does so independently of PHP (or any other language) by hooking directly into Apache. This makes it possible, for example, to manage sessions directly from within HTML pages, which, in turn, could mean lower overhead when processing pages. iConductor costs $1,295.00 and is available directly from Farpointer at http://www.i-conductor.com.

php|a

5

PHP-WIN

PHP-W WIN

Reviewed For You

Davor’s PHP Editor By Arbi Arzoumani

F

or a vim addict (recently converted to using gvim and the open-source CREAM extension) like myself, GUI IDEs are something to look at with an air of suspicious resentment. On one hand, they often look easy to use and user-friendly (something that can’t really be said about vim). On the other, however, my shell junkie friends and I fear that, as soon as you need to do something out of the ordinary, they will turn into rigid monsters that won’t allow you to move past their pre-set functionality. Luckily, I’ve had a chance to find out that it’s not necessarily always the case. After having been scarred for years by Microsoft’s Visual Studio IDE, I had a chance to use Zend Studio, which I found extremely powerful (particularly in its debug functionality, which are very valuable), although made a little slow by the fact that it’s based on Java. Still, even for its price, Zend Studio is extremely good—and works across multiple platforms. A few weeks ago, I came across Davor Pleskina’s PHP Editor (DPHPE from now on), which is available from his website at http://www.pleskina.com. Davor is a great guy (he’s also the author of this month’s article on catalogue management), and he suggested that I give his IDE a try. Firing Up DPHPE is extremely easy to install, although it doesn’t come with a setup program of its own. You do have


to manually copy the files from the ZIP file you can find at Davor’s website into a folder on your PC and create the necessary shortcuts. I can’t see this as being a huge problem, since it’s safe to assume that you probably have some level of confidence with the OS if you’re interested in this product. Still, creating a setup program is not a particularly challenging endeavor these days—there are plenty of tools for that—and a more guided approach to installation would probably benefit most users. To give you an example, during my first attempt at running DPHPE, I simply executed the application’s main executable from directly within WinRAR (my compression application) without copying any supporting files. The results were not catastrophic—the application was running fine for the most part—but I had lost access to some functionality, such as keyword autocompletion.

DPHPE is also able to maintain a list of all the variables that are currently in scope in your script... Once installed, DPHPE takes up very little space on

6

PHP-WIN

Davor’s PHP Editor

your hard drive, which luckily also means that it uses very little RAM when it has to run. As much as this doesn’t seem to be a big problem these days, if your machine is running a number of different services at the same time, the last thing you need is another application that gobbles memory.

Another well-developed feature of DPHPE is the ability to create and maintain projects. Basic Features Where do I begin? DPHPE has a wide array of different features that are undoubtedly helpful for any developer. Most impressive of all, these features are implemented in a simple and efficient way, so that they actually help without getting in the way (at least most of the time). The editor that comes with the application features syntax highlighting (for HTML, PHP, Javascript and CSS), and its aspect is fully customizable—both


from the point of view of the fonts used and of the colours in which the syntax is rendered. Also featured in the editor is the ability for auto-completion of keywords, which is particularly useful for a language as complex as PHP. It is also possible to have the application create a list of functions defined in your scripts, although this doesn’t always work well—for example, I had a comment that contained the word “function”, and the editor mistakenly recognized it as a declaration. DPHPE is also able to maintain a list of all the variables that are currently in scope in your script (although, unfortunately, it is unable to extend this capability to any include files you may be using) and to automatically pop up a window that allows you to choose a variable whenever you type the dollar sign. This feature tends to be a bit annoying after a while, but it can be turned off and the variable list can be used manually as needed. My only real beef with the editor is that I couldn’t find a way to word-wrap my long lines of code. Not everyone uses this feature, but I find it very convenient and not having it makes it a bit difficult to use the editor in a comfortable way.

7

PHP-WIN Project Management and Web Preview Another well-developed feature of DPHPE is the ability to create and maintain projects. A project is simply a repository of files but, contrary to a folder, it also possesses a number of properties that make it possible, for example, to preview entire web projects at the click of a mouse. What’s most interesting, however, is that the concept of projects lays the foundation for a number of future expansions, such as encoding, CVS or SourceSafe compatibility, global search and replace, and so on, that could all be accessible at the click of a mouse. Speaking of web preview, if you have Internet Explorer installed on your machine (which you must have, unless you’re running some ancient version of Windows), DPHPE is capable of interfacing to it and visualizing your PHP scripts by executing them through your web server. If you prefer, you can also launch IE separately rather than within the editor. Unfortunately, DPHPE does not feature an internal debugger—and I personally think


Davor’s PHP Editor that this should be a very important future enhancement. Still, this application offers a great deal of functionality at a very convenient price—it’s free. What’s more, there is no big corporate presence behind it, so that it really evolves following the needs of its users, and Davor does a great job of listening. php|a

8

FEATURES

FEATURES

Performance Management Opportunities By Zeev Suraski and Brad Young Why Focus on Performance Management? A look at the various alternatives that exist in today’s complex IT world to improve server performance for sites that run PHP.

T

o start, let’s consider the reasons why we should be concerned with Performance Management. In a nutshell, it can be summarized with two simple goals: 1) Save Money 2) Happy Customers Although this may seem obvious, or even trite, it is important to keep in focus when evaluating and deploying performance-related products. To take it one level deeper, let’s understand the issues faced by IT managers and webmasters today:

· High expenses for hardware and software purchases to support http site traffic

· High expenses for database server hardware and software

· IT administration time and money spent to reconfigure and maintain server farms

· Low customer satisfaction resulting from unreasonable latency time and load time With these issues in mind, we can now more clearly attack the question of how to achieve improvements for our projects.


As Yogi Berra famously stated, “It’s hard to make predictions, especially about the future.” For PHP, however, there are some things that are not so difficult to predict. Options For Performance Management There are many different ways through which you can squeeze more juice from your limited hardware server resources. These options include hardware, software, and networking alternatives. We’ll highlight each of these options below. Note that these alternatives are not mutually exclusive. Hardware Options

More Hardware - The most common indicator – aside from emails from angry site users – that raises a red flag about performance issues is maxed out CPU utilization. And when this happens, an obvious but costly solution is simply to buy more hardware. If you have two dual-CPU http server machines whose CPU’s

9

FEATURES


are maxed out, then adding a third machine, with load balancing, will alleviate some of this load stress. It is worth noting that this solution works for steady state performance issues, but does not scale well for spiked performance issues. For example, if your site experiences dramatic increases in requests at specific times of the day, you’ll need to add enough hardware to cover the ‘worst-case’ load. (Actually, it is more accurate to call this the ‘best-case’. We want more traffic on our sites, after all.) Smarter Hardware – Similar to adding more hardware, adding smarter hardware can provide performance improvements. This includes the more obvious steps of having machines with faster CPUs, or more CPUs per machines. Beyond that, there is also hardware components such as disk drives with built-in memory caching that also help improve hardware performance. Hardware Devices – Another hardware option to improve performance is to utilize hardware devices that have built-in algorithm circuitry. For example, performing SSL encryption using software on your http server adds a significant stress to the CPU load. You can add a hardware card that performs this encryption without requiring any CPU time, thus lowering the effective load. Similarly, this can be done for gzip compression as well. Faster Code

It is clear that if your code is faster and more efficient, you’ll be able to handle more requests with the same hardware. Let’s explore how we can make code run faster. C Libraries– The massive adoption of PHP has come about mainly due to its ease of adoption – both from a cost perspective as well as a development perspective. But while PHP speed has improved tremendously, it is a fact that C code runs faster. So if you develop some of your more algorithmically-complex functions in C and call out to them from within your PHP scripts, you can speed up your code. Of course, by doing this, you give up some of the flexibility, transportability and ease of use that brought you to PHP in the first place. Write Better Code – Better coding practices, while tiresome at times to adopt, pay off in the long run. Always stay up to date and aware of the best ways to use PHP by following email lists from php.net and reading tutorial materials at sites like PHPbuilder.com and at the Zend Developer Zone (devzone.zend.com)—not to mention the pages of php|a, of course! Code Acceleration – PHP is an interpreted scripting language. This means that each time a PHP script needs to run, the core Zend Engine will parse that file and compile it in memory, then execute it. Since the PHP scripts themselves typically don’t change from December 2002 · PHP Architect · www.phparch.com

request to request, this parsing and compilation phase is repetitive and wasteful. Code Acceleration products available today can eliminate that redundancy by maintaining the compiled version of the file in memory. Output Content Caching

Imagine that your friend asked you what the headline of the New York Times was today. To respond, you go outside, walk two blocks to the nearest news-stand, read the headline of the New York Times, then come back and tell him what that headline says. (We leave it to you to understand why your friend would ask this, but just go with the analogy). Now, imagine that another friend calls you and asks the exact same question. Would you repeat the journey down to the news-stand? Unless you are madly in love with the salesperson, probably not. You are smart enough to identify that this new request is identical to the previous request, and hopefully you are also smart enough to remember what the headline was.

PHP is an interpreted scripting language. This means that each time a PHP script needs to run, the core Zend Engine will parse that file and compile it in memory, then execute it. Well, unfortunately your http server is not as smart as you, and it, figuratively speaking, is probably making way too many trips down to the news-stand (read: database server). This is where content caching comes in. The idea of content caching is straightforward: your server works hard to generate some results to a specific request. Then it keeps those results on hand, and if it gets a similar request, it re-uses the previously generated results. It is interesting to note that caching takes place at the http server level and reduces load on that server, but an even bigger load savings is experienced on the database server, which sees dramatically less SQL requests coming in. The concept is simple enough, but with all things programmatic, the devil is in the details. The challenge mainly boils down to: · Cache Hit Identification – How does the cache determine if this is indeed a similar request, or is it a different request (i.e. Is my second friend asking for the same headline, or is he asking

10

FEATURES

Performance Management Opportunities Figure 1- Content Caching Schematic

about the New York Post, or is he asking about yesterday’s New York Times, or is the headline different for him because he gets the national edition instead of the local edition)? · Cache Management – How does the cache manage the information results that are onhand, purge old information, and prevent over-use of system resources (i.e. I can’t remember every headline of every newspaper from every day without my head exploding, so which is the best to forget)?

Now let’s look at a few ways of achieving content caching: File-level Content Caching – If you have web pages that appear the same to many users, you can store the entire results of the URL request coming in to your http server, as shown in Figure 1. For example, imagine a page that lists news headlines, such as cnn.com. New headlines are added, perhaps at a rate of one new headline per 15 minutes. CNN could cache the headlines page or at least 5 minutes. Now, if 1,000 page view requests come in during

Figure 2 - Partial Page Caching


11

FEATURES


that 5 minute period, only one actual PHP execution and one database access will occur, a dramatic decrease compared to the 1,000 that would otherwise occur. All the other requests are served directly from the cache, as if it was a static html file. And by having a time limit to this cached file, the worse case scenario Figure 3 - Database Intermediary

“Welcome, UserName” problem. In our headline listing page, the main portion of the page is the same for each user, but now we add a personalization message at the top of the page. We can no longer cache the entire file, unless we don’t mind that users named Judy, John, and Jade will get a page that says “Welcome, Joe”, the first person to request the URL.

PHP 5.0, with its new Zend Engine 2, will include new object handler capabilities that will not only make coding easier, it will also speed up application performance.

for content freshness would be that it would take a new headline 4 minutes and 59 seconds to make it to the site. When exploring solutions, you’ll want to make sure that there is sufficient flexibility for determining what should be considered a ‘cache hit’. Besides the basic URL address, you might also take specific cookie variables, GET parameters, or global http variables into consideration. For example, you might want to cache a different version of your home page for each geographic region, or type of user. Partial PageCaching – Solving the “Welcome, UserName” Problem– The benefits of file-level content caching are clear. But often, some portions of the page are personalized on a per-user basis. Let’s call this the

The solution here is partial page caching. We still can cache the majority of the page, which in this case also uses the most resource-intensive database queries. This is done at an API level, in which the caching of data segments is maintained and reused programmatically, as shown in figure 2. Database Intermediary Caching– A third option for caching of results involves separating your PHP code from the database, as shown in Figure 3. Instead, have a separate timed job that pulls content from the database and creates chunks of html files that the PHP scripts then can read in the output results that they are creating. This solution adds some complexity with regards to system configuration and code dispersal, but the results will sometimes make it worthwhile.

Connect with your database Publish your data fast with PHPLens PHPLens is the fastest rapid application tool you can find for publishing your databases and creating sophisticated web applications. Here’s what a satisfied customer, Ajit Dixit of Shreya Life Sciences Private Ltd has to say: I have written more than 650 programs and have almost covered 70% of MIS, Collaboration, Project Management, Workflow based system just in two months. This was only possible due to PHPLens. You can develop high quality programs at the speed of thinking with PHPLens

Visit phplens.com for more details. Free download. December 2002 · PHP Architect · www.phparch.com

12

FEATURES


Database Optimization

Data Models and SQL Tuning – As programmers, we often find ourselves focusing on our area of expertise – coding. However, even the fastest optimized algorithm as written in PHP can’t make up for the time wasted by poorly written SQL statements or queries to databases that are not modeled or indexed properly. Don’t fall into this trap. As mentioned earlier under the subject heading of “Write Better Code”, take the time to learn about how to get the most out of your database server. If you have a DBA in house, use his or her expertise. As with content caching, a little effort can go a long way when you tweak your database usage.

Also, identify the resources and infrastructure you have available. Once you measure these parameters, selecting the various products or methodologies mentioned above becomes much easier. Enterprise Performance – Future Directions As Yogi Berra famously stated, “It’s hard to make predictions, especially about the future.” For PHP, however, there are some things that are not so difficult to predict.

· First, Apache 2.0 adoption may start to grow,

Data Compression

Gzip Compression – “Performance problems” is a big umbrella. Sometimes, the performance issue is not necessarily server throughput or CPU utilization, but rather end-user experience of download time. Slow download may result from either large data transfer, or overutilization of server bandwidth. In both cases, you will improve end-user performance experience by compressing your data. Most browsers today can receive compressed contents, and will decompress automatically. Using output compression is as simple as setting zlib.output_compression = on in php.ini. Browser detection will determine if gzip support is built in, or if the original plain text should be sent. However, it is important to note that this adds some additional CPU burden to your server, which must perform a gzip compression for each result being delivered. This can be overcome by using a compression solution that is integrated with content caching. In this case, the compressed version is cached, and not regenerated each time. Network Infrastructure

While it isn’t directly related to PHP dynamic content, it is worth noting that network optimization solutions, such as those provided by Akamai , can boost content delivery time. This can be especially useful if image download and delivery time is your performance bottleneck. Which Should You Choose? As stated in the opening of this article, these options are not necessarily mutually exclusive. So, how do you make choices on how to boost your performance? Well, in the words of the anthropologist Margaret Mead, “Always remember that you are absolutely unique, just like everyone else.” First, identify what specific problems you are having.


·

·

when the Apache stability as well as the PHP stability on top of Apache 2.0eventually reaches the necessary level. We may see some boosts to performance from Apache 2.0’s multithreaded capabilities, on platforms such as Solaris. PHP 5.0, with its new Zend Engine 2, will include new object handler capabilities that will not only make coding easier, it will also speed up application performance. In the database world, newer extensions and support for MySQL 4 and Oracle will likely give another shot in the arm for database throughput. Why Performance Management Matters

We opened this article explaining the benefits that Performance Management brings you and your organization: Save Money and Happy Customers. But the importance runs even deeper. Collectively, the PHP community is proving every day that PHP is ready for front-line, enterprise-level applications. Implementing performance management solutions such as those highlighted here bring us further towards the recognition from the marketplace that PHP is here to stay. php|a

Zeev Suraski has been working for over five years on the PHP project. Along with Andi Gutmans, he started the PHP 3 and 4 projects and wrote most of their infrastructure and core components, thereby helping to forge PHP as we know it today and attracting many more developers to join the movement. Zeev is a co-founder and CTO of Zend Technologies Ltd, the leading provider of development and performance management tools for PHP-enabled enterprises. Brad Young leads the Product Management team at Zend, and has over 15 years experience in on-line database and XML systems deployment. You can reach both Zeev and Brad via Zend's website at www.zend.com.

13

FEATURES

FEATURES

Curses! Writing Cool Shell Applications With PHP By Marco Tabini While PHP is most widely used as a Web development language, I find that it works very well as a shell tool, too. In fact, I’d have to say that PHP is my favourite shell language, and that I prefer it over typical shells or even Perl (although the latter is rather unbeatable when you have to process large data files).

I

n a world of web-based management systems and graphical user interfaces, it’s comforting to find out that shell programming is still regarded as an important piece of the IT puzzle. Don’t get me wrong—I’m not saying this merely out of nostalgia for the times when I had to make do with 48kB of RAM for operating system, program and data, or had to wrestle a few cycles of program execution out of a mainframe. Text-based applications are still very useful in a number of situations, particularly when simplicity is called for or a GUI would be oppressively slow to deal with (such as when you’re trying to manage a web server that is heavily loaded and cannot spare much bandwidth for you). Shells, particularly (but not exclusively) in the UNIX world, are very much alive and kicking, and have advanced significantly in complexity and functionality over the past few years. nCurses - A Library For Cool Shell Apps Think “shell application” and the image of a dull, black-and-white text screen comes to mind. Luckily, most terminals are actually capable of a lot more than just displaying text: they provide a variety of colours and effects (such as underlined or bold—”bright”— text), mouse compatibility and much more. It’s normally rather difficult, however, to take advantage of these capabilities for a number of reasons. First, not all terminals are created equal—there are several different stan-


dards available, each with its own commands and capabilities. Second, because terminal systems have to work over simple serial communication lines, the commands they support are often intricate combinations of ASCII characters. Since shell applications are usually written as a quick and easy solutions to simple problems, the developers rarely have the time and patience to bother with adding the “frills” to their interfaces. Those of you who, like me, have spent their youth wasting their allowances on telephone bills as a result of the expensive hobby known as “BBS”, however, know what terminals are capable of. Remember the cool menus, the dazzling animations and the colourful interface? Compare that with your average shell application and, well, I’d take the BBS over it every day. On UNIX systems, the capabilities of terminals are stored in the termcap database, a file usually found in the /etc directory that is used by applications such as the original vi editor. Termcap contains a description of each terminal that is recognized by the system (for example, “dumb” or “ansi-compatible”), together with the commands it supports. Termcap, however, is not in a very “developer-friendly” format. In fact, I think it’s fair to say that using it in REQUIREMENTS PHP Version: Latest PHP CVS Build O/S: nCurses-compatible O/S Web server: N/A Support Tools: nCurses Library

14

FEATURES


an application with no outside help would require a major effort, since you would only have to interpret its contents, but also write the necessary functions to handle tasks like cursor positioning, text attributes, window management, and so on. It was probably out of desperation that, around 1980, a student at UC Berkley named Ken Arnold developed a series of routines that provided an essential framework for terminal interface development and collected them in what became known as the “curses” library. Curses then became the basis for Rogue, possibly one of the first-ever Role Playing Games, initially developed for BSD Unix but eventually ported to pretty much any operating system ever conceived. If you have never played Rogue, I suggest you give it a try— not only it’s a thoroughly enjoyable and never repetitive game, but it also gives you an idea of how engaging a simple text-based user interface can be. You can find Rogue in RPM format at : http://www.rpmfind.net/linux/rpm2html/ search.php?query=rogue I should warn you that you will find it highly addictive; your productivity will probably drop to zero for a while, but you’ll have lots of fun in the bargain.

• When you need to deploy a web-serving framework capable of delivering millions of views and transactions per day... • When you need to construct complex document and task workflows that match your organizations' needs... • When you need to create a world wide streaming network capable of 99.99% up time... • When you need to protect your country's sovereignty by avoiding "random" changes in software licensing that can affect the price of each transaction your citizens make... • When you need the level of security that only expertise and transparency can provide you...

The Curses library was picked up by Bell Labs, which included it in its System III version of UNIX, introducing several changes to termcap in the process. Termcap’s new format, called terminfo, offered more functionality. Additional development was done through the 1980s, and in 1993 the package was finally renamed ncurses. Ncurses and PHP The ncurses library provides a number of features, such as: · Formatted input and output on a variety of terminals · Support for colours and various text styles, such as underline, bold, and so on · Support for windowing · Support for mouse input PHP support for the ncurses library is still very much in its infancy, and is based on the GNU implementation of the library, available via FTP at: ftp://ftp.gnu.org/pub/gnu/ncurses

• When you need Single-Sign-On across multiple vendors serving a diverse constituency... • When you need to reduce your reliance on proprietary software solutions and realize the promise of your existing investments... The Tribal Knowledge Group's demonstrated credentials in the Open Source Software community enable them to provide you with decades of software engineering, architecture, security and strategy expertise via an unprecedented think-tank of Internet technology and open-source experts. Contact us at [email protected] to learn how the Tribal Knowledge Group can help maximize your existing investment for the future.

Proven Open Source Expertise


15

FEATURES It was originally introduced by Hartmut Holzgraefe and Georg Richter with version 4.1.0 of PHP and is still considered highly experimental, to the point that the official PHP documentation covers the ncurses functions only very quickly and in a still incomplete fashion. Luckily, the ncurses extension functions follow their original counterparts very closely, which actually makes the process of creating an ncurses-powered PHP script very easy (this is something that the authors of the extension must have undoubtedly known, so that they have—correctly, in my opinion—put all their efforts in building out the extension). Ncurses is actually supported on a number of platforms, including all the UNIX derivatives, cygwin and BeOS. Obtaining the Right Version of PHP The problem with making ncurses run on your server is that, in order to gain access to enough functionality, you will have to download the most recent version of PHP from the public CVS tree. There is no way around this problem, as the stable releases available to this point do not yet include the latest code, which is unfortunately necessary to perform most of the basic ncurses tasks. Luckily, obtaining the latest version of PHP is not difficult at all. You can get it directly by issuing a couple of CVS commands, which are clearly explained in the manual:

cvs -d :pserver:[email protected]:/repository login cvs -d :pserver:[email protected]:/repository co php4

If all goes well, you should end up with a sub-folder of your current directory called php4. Simply cd into it and configure PHP as you would normally, ensuring that you include --with-ncurses to your invocation of configure, and that you compile the CLI version of PHP—after all, I’m afraid that you would have little use for the ncurses extension in a web application. Our First ncurses Application It’s now time to test whether the installation went smoothly by executing the simple PHP script that you can see in Listing 1. This script is designed to be run from the shell. As such, it includes a “shebang”, that is, a specially-formatted comment (always the first line of the script), that instructs the shell as to which interpreter this file should be executed through. This approach works only on UNIX-type shells, however, so if you’re running Windows you’ll have to use the cygwin shell rather than the DOS command prompt. December 2002 · PHP Architect · www.phparch.com

Curses! Writing Cool Shell Applications With PHP If you do not receive any errors, you should see the words “Hello World!” written in red text over a cyan background appear at the top of your screen for a few seconds and then disappear. If something goes wrong, there’s a good chance that your shell will become unusable—don’t panic, this is quite normal, as we’ll see in a few moments. As you can see, even a simple Hello World application requires quite a bit of code if you write it using ncurses. You not only have to initialize the library using the ncurses_init() function, but you have to create a combination, or “pair”, of each foreground and background colours that you want to use in your application. This, in turn, will only work if the terminal you are using supports colours to start with, and the ncurses_has_colors() function provides the script with that information. Even writing a string to the screen requires a call to a specialized ncurses function, ncurses_addstr(). The latter is actually part of a family of functions that make it possible to manipulate text on the screen. Incidentally, you will also have noticed that it’s necessary to forcefully refresh the screen in order for any updates made to it to become visible. This particular aspect of ncurses programming is actually designed so that only the minimum amount of commands are sent to the screen, and only when necessary—thus reducing the amount of data sent to the terminal by as much as possible. With today’s fast connections, this may sound a bit ridiculous, but you have to understand that ncurses must be able to work efficiently even over slow serial connections—and a bit of optimization never hurts. Thus, for the most part, an application writes to a “virtual screen” and then issues a call to ncurses_refresh() when all of its updates are complete and it is ready to show its output to the user. Finally, the ncurses library must be explicitly terminated by calling ncurses_end(). This is necessary because ncurses captures the terminal and creates its virtual screen over it. As a result, exiting from a script without also terminating ncurses results in a terminal that, in most cases, is completely unusable. The problem of course, is that sometimes scripts end unexpectedly, as it is often the case when errors occur. For this reason, it’s always a good idea to create your own error handler that explicitly calls ncurses_end() before aborting your script. This will not protect you from all kinds of problems (parse errors will still cause PHP to exit without calling your handler, for example), but it will take care of at least some cases. With the exception of windowing functions, which we’ll examine in the next section, all ncurses functions return true if an error has occurred (they actually return an integer value different from NCURSES_ERR, which is defined as zero).

16

FEATURES

Curses! Writing Cool Shell Applications With PHP Windowing

One of the most interesting characteristics of ncurses is the ability to create and manage multiple windows. Although they cannot overlap each other, each window possesses its own “virtual screen” and can be modified and refreshed independently of its other counterpart on the terminal. As an example, let’s take a look at Listing 2, which introduces a number of new ncurses functions. The first one is ncurses_getmaxyx(), which returns the dimensions of the virtual screen. You have probably noticed that, in this function, the height is listed in the function call before the width—something that may appear unnatural to someone who is used to working with GUI systems. All ncurses functions follow this convention, however, and it does take a while to get used to it. Also, the call to ncurses_getmaxyx() at the beginning of Listing 2 uses a constant called STDSCR. This constant describes, by convention, a pointer to the main virtual screen, which is, for all purposes, a window of its own right. In fact, most of the ncurses functions that manipulate the virtual screen also have a windowspecific counterpart—for example, ncurses_addstr() and ncurses_waddstr(). In most cases, the ncurses library implements the virtualscreen version of these functions as simple macros to the window versions with the hardcoded STDSCR parameter. The ncurses_newwin() function is used to create a new window. Its syntax is as follows: resource ncurses_newwin ( int $height, int $width, int $y, int $x );

Its result is an ncurses resource that you can use in subsequent calls to other ncurses functions that can manipulate the contents of your newly created window, such as ncurses_wborder, which—you guessed it—creates a border around the frame of our window: int ncurses_wborder ( resource $win_resource, int $left_side, int $right_side, int $top_side, int $bottom_side, int $top_left_corner, int $top_right_corner, int $bottom_left_corner, int $bottom_right_corner );


With the exception of the first parameter, which identifies the window where the border is to be drawn, each of the other parameters represent the characters that the function must use to build the border. If you leave all these parameters as NULL or zero, ncurses_wborder() will create a standard (and classy) border around the window. If, for example, you call the function as follows: ncurses_wborder ( $win1, ‘|’, ‘|’, ‘-’, ‘-’, ‘+’, ‘+’, ‘+’, ‘+’ );

You will obtain a slightly less pleasant—but just as functional—border. Writing into a window is performed through a number of functions. The simplest one is ncurses_waddstr(), which is functionally equal to ncurses_addstr(), with the exception that it takes a window resource as the first parameter. In the case of Listing 2, however, I have used a slightly more complex version of it called ncurses_mvwaddstr(), which at the same time moves the cursor to a pre-defined position and prints out the string passed to it: int ncurses_mvwaddstr ( resource $win_resource, int $y, int $x, string $str );

As you may have noticed, the x and y coordinates passed to this functions reference the top-left hand corner of the window—not of the screen. There is, naturally, an equivalent of ncurses_mvwaddstr() that works with the virtual screen and is called ncurses_mvaddstr(). This, in turn, is functionally equivalent to calling ncurses_mvwaddstr() as follows: ncurses_mvwaddstr ( STDSCR, $y, $x, $str );

Finally, we come to the visualization of windows, which is achieved by calling the ncurses_wrefresh() function. Keep in mind that ncurses does not handle windows the same way a GUI system like

17

FEATURES Windows or XWindows would; this means that the overlapping of windows and features like z-order are not normally provided to the developer (although another portion of ncurses called panels, which is also being implemented in the PHP extension, does provide exactly this functionality). Thus, wrefresh() should simply draw a particular window on the screen, as if it were in focus and with complete disregard for whatever other windows may overlap with it. I say “should” because in reality different versions of ncurses behave in different ways—as such, it’s best to either use only tiled windows, or to use one window at a time on top of the virtual screen. Thus, calling wrefresh() will cause a window to be drawn ontop of the virtual screen, in our cases advising us that its contents are rather messy. Another interesting characteristic of ncurses is that it provides no scrolling capabilities. When you reach the end of the screen (or of a window), text is simply discarded. Depending on the type of application that you are writing, this could be a major problem—but it can be easily solved as we’ll see later on. A Bit of Colour and a Sprinkle of Fairy Dust As marketing people have known for years, colour makes all the difference. In a well-designed interface, proper colour management can make any application easier to use (except for vim, which was obviously written by aliens) and more pleasant to the eye. The ncurses library provides an extensive set of functions that deal with the aspect of text—not only with its colour, but also with its intensity and style. For example, Listing 3 produces text in quite a variety of styles. Because colours can only be rendered on some terminals, the ncurses extension provides the ncurses_has_colour() function to determine whether colours can, in fact, be used on the current terminal. Also, colour rendering has to be enabled by calling the ncurses_start_color() function. The ncurses library, as I mentioned above, handles colours by means of “pairs”, each representing the combination of a foreground and background colour. Each pair is defined using the ncurses_init_pair() function: int ncurses_init_pair ( int $pair_id, int $foreground_colour, int $background_colour );

The ncurses extension provides a number of pre-set colour definitions, which are shown in Figure 1. On some terminals, it is also possible to create custom December 2002 · PHP Architect · www.phparch.com

Curses! Writing Cool Shell Applications With PHP colours by changing the RGB values associated with a particular pre-defined colour (you could finally have your own opportunity to invent the “greenest blue” ever made!) through a call to ncurses_init_color(), whose syntax is as follows: int ncurses_init_color ( int int int int );

$color, $rgb, $green, $blue

A call to ncurses_can_change_color(), which takes no parameters, will tell you whether you can, indeed, change the appearance of a particular colour. Figure 1 NCURSES_COLOR_BLACK NCURSES_COLOR_WHITE NCURSES_COLOR_CYAN NCURSES_COLOR_RED NCURSES_COLOR_YELLOW NCURSES_COLOR_BLUE NCURSES_COLOR_MAGENTA NCURSES_COLOR_GREEN

Black White Cyan Red Yellow Blue Magenta Green

Attributes are handled in a very similar fashion to HTML. They can be turned on using the ncurses_attron() function (or ncurses_wattron() for a window), or off through a call to ncurses_attroff(). A list of available attributes is shown in Figure 2. Figure 2 A_NORMAL A_STANDOUT A_UNDERLINE A_REVERSE A_BLINK A_DIM A_BOLD A_PROTECT A_INVIS

Normal font Standout (reverse) Underlined Reverse screen Blinking Text Dimmed Bold Protected Invisible

Similarly to colours, there is a way to determine whether a particular attribute is supported by the current terminal. The ncurses_termattrs() function takes no parameter and returns a logical OR of all the attribute values that the terminal can handle.

18

FEATURES

Curses! Writing Cool Shell Applications With PHP Listing 1

1 #!/usr/local/bin/php 2 3

Input Management As I’m sure you can imagine, a good user interface without input capabilities is a bit like a war with no enemy—you could make one but it would be rather dull and quite pointless. The ncurses library offers a number of facilities for both keyboard and mouse input, although we will only focus on the latter as far as this article is concerned. The most commonly used form of keyboard input is ncurses_getch(), which retrieves a single character from the input stream. Under normal circumstances, that character is also outputted to the screen, but you can prevent that from happening by calling the ncurses_noecho() function. For example, take a look at Listing 4, which shows a simple script that can be used to control the Apache web server through a series of simple commands. As you can see, there are a couple of interesting functions. First, I’ve created a simple msgbox() routine


that takes an arbitrary amount of text and displays it in an ordered fashion on a window of its own. This corresponds to a simple dialogue window in a GUI environment. You will notice that I have used the wordwrap() function to ensure that the text is always properly centered and never interferes with the window’s borders. The lack of a couple of ncurses functions (in particular wsyncup() and wsyncdown()) in the PHP extension make it slightly awkward to manage windowing properly, so that the application is forced to manually refresh the screen every time a dialogue box is displayed. The ncurses_getch() function is called in an infinite loop until the user presses the “q” key and exits. The result value of ncurses_getch() is not, however, a character, but an integer value. This lets you manage “special” characters, such as Page-up and Pagedown, more easily and conveniently—but you will have to convert the integer value to a string if you want to interpret it as a character. More complex input can be

19

FEATURES


performed by calling the ncurses_instr() function, which reads an entire string from the input and returns it. It’s interesting, at least in my opinion, to note that even a simple script like this one can be set as the default command for a particular user account, thus depriving it of shell access but still giving it the opportunity to perform certain operations—in this case, stopping and starting Apache. A More Complex Example Let’s now move on to a more audacious attempt at using the ncurses library to produce a complete shell-

based application. Listing 5 contains a self-contained text file viewer that allows for up and down scrolling (both line by line and page by page), can split and display lines of text that span multiple lines (without using wordwrap, this time) and displays a status line at the bottom of the screen. This script introduces the ncurses_move() function, which is used to move the cursor to a particular location (in fact, ncurses_mvwaddstr() is essentially the equivalent of a call to ncurses_move() followed by a call to ncurses_waddstr()). You’ll notice that the most complicated action performed by this script is the tracking of the position of the screen’s output within the text file. For the script to

Listing 2 1 #!/usr/local/bin/php 2 3

Listing 3 1 #!/usr/local/bin/php 2 3

Listing 4: Continued From Page 23... 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155

main_menu(); while (true) { $c = ncurses_getch(); $s = chr ($c); switch ($s) { case '1' : msgbox (shell_exec ("apachectl start") . "\n\nPress a key to continue"); main_menu(); break; case '2' : msgbox (shell_exec ("apachectl stop") . "\n\nPress a key to continue"); main_menu(); break; case '3' : msgbox (shell_exec ("apachectl restart") . "\n\nPress a key to continue"); main_menu(); break; case 'Q' : case 'q' : ncurses_end(); exit; break; } } ?>


24

FEATURES

FEATURES

SNMP Management With PHP By Harrie Hazewinkel In the last decade, the importance of distributed services has become clear, as organizations have deployed networks to automate their business processes. As a result, organizations now depend on their networks and the related services, and any minute of down-time costs money. Not only personnel cannot perform their work during down-time, but many automated services, such as telesales, for instance, stop working. If an organization monitors its network, it can detect problems before they become a problem, increasing productivity and reducing personnel costs. This article provides a brief explanation of the SNMP protocol and explains the SNMP functionality that is available through PHP.

Simple Network Management Protocol

T

he Simple Network Management Protocol (SNMP) provides a framework for network management. SNMP enables access to network management information to make, for instance, status monitoring, faultdetection and configuration possible. Since its introduction, SNMP has been used heavily to perform monitoring and fault detection tasks. There has been little use of SNMP for configuration purposes, though, since it lacks a proper security protocol. However, the latest SNMP framework (version 3) defines security functionality by means of authentication, privacy and access control. These features protect the network against threats such as: 1. Modification of an SNMP message in transit by an unauthorized SNMP entity. 2. Modification of the SNMP message stream by changing the order, delaying or replaying messages. 3. Masquerading of SNMP entities through the appropriation of another SNMP entity’s identity. 4. Disclosure of management information that is December 2002 · PHP Architect · www.phparch.com

transferred between SNMP entities (users). 5. Eavesdroppers that attempt to capture the management data for other “evil” purposes. 6. Unauthorized access to particular management information. If you compare these capabilities with earlier version of SNMP, which only offered a single clear text password authentication method and no encryption, you’ll see that SNMPv3 represents a big step forward for the protocol’s security features. What’s more, SNMPv3 defines three different security levels: 1) No authentication and no privacy 2) Authentication and no privacy 3) Authentication and privacy It also allows for the use of different authentication and privacy protocols, thus enabling SNMP to evolve over time as new and better security mechanisms are REQUIREMENTS PHP Version: PHP 4 (4.3.0 for SNMPv3 support) O/S: Any Web server: Any Support Tools: N/A

25

FEATURES

SNMP Management With PHP Figure 1

defined. The SNMP framework

An SNMP manager requests management information from an SNMP agent. The manager invokes an SNMP request on an agent and the agent collects the requested information from its MIB and produces a response for the manager. The manager interprets the returned information and can use it for logging, status reporting or for automated actions. like initiating alarms. A typical example of this would be a polling system in which periodically traffic counters are collected by the SNMP manager.

Figure 2

An SNMP agent initiates a notification to a manager. Notifications are used when an agent detects an unusual or anomalous situation on its own, such as when an interface on the host goes down. Upon detecting the fault, the SNMP agent informs directly the SNMP manager of it, so that the latter can take the appropriate action, such as initiate an alarm.

Figure 3


SNMP is built around an asymmetrical framework of many SNMP agents and one (or a few) SNMP managers. An SNMP agent resides in a remote host or device and is connected to the actual hardware that needs to be monitored. It provides access to the hardware via managed objects through an interface known as Management Information Base (MIB). The agent responds to queries from managers that request information from its MIB. A manager can therefore create a complete and coherent picture of the network’s status by requesting specific management information from multiple agents. The Management Information Base The Management Information Base (MIB) contains the managed objects under management by an agent. Briefly explained, the MIB is a virtual information store composed of one or more MIB modules. Each MIB module implements a specific set of management information, which is itself organized in an ordered tree. Only the leaves of this tree can contain real data, and then only if the object/node is defined as an ‘OBJECTTYPE’ (we’ll get to that in more detail later). The intermediate nodes used for creating the hierarchy do not actually contain any information, but provide the tree with a logical structure. Figure 3 illustrates an MIB tree example that is a part of the SNMPv2-MIB as defined in RFC 1907 (which can be downloaded from http://www.ietf.org/rfc/rfc1907.txt, if you’re interested in reading the complete MIB module specification). The tree depicted in this snapshot contains three objects (sysName, sysContact and sysLocation) representing scalar variables, as well as two objects ordered in a table under the sysOREntry node. The table objects are ordered by column (sysORID and sysORDescr), while each row is uniquely identified by an index. The indexes used in Figure 3 are 2 and, therefore, the table contains 4 values (2 columns * 2 rows). Each node in the MIB tree has a human readable name and is ordered by number. A specific node is described by an Object Identifier that combines the numbers of the branches along the path starting from the root. To find a particular leaf in the MIB, both an exact and a next search can be performed. The exact search requires that the name of the leaf being searched for exactly match what is saved in the data store, while the next search returns the first lexical graphical higher leaf. Figure 3 depicts a sample walk. Each arrow indicates

26

FEATURES the “next” of a particular object. To indicate that there are no more “next” values in the MIB tree, an exception called ‘EndOfMIBView’ is used. Structure of Management Information The language in which MIB modules are written is called Structure of Management Information (SMI). SMI was originally based on the Abstract Syntax Notation 1 (ASN.1). Let’s take a look at a small portion of this language - just enough to understand the creation of an MIB tree. The entities defined in a tree must all conform to the following standard format: ‘::=’ ‘{‘ ‘}’

The is used as a human readable identification of the object/node in the MIB tree. The provides specific information of the object. In the excerpt, all three definitions are of the ‘OBJECTTYPE’ kind, which includes the syntax of the object, the type of access granted to it (read/write), its status and its description. The provides the hierarchy information. Typically it consists of a single name and a single number. The name indicates the parent node in the MIB tree and the number indicates the current position in the tree with respect to its ancestors. For example, in Figure 3, the definitions of ‘system’ has the number 1 and the parent node is ‘mib-2’. The ‘sysContact’ entity defines the contact information of the particular system on which the MIB is defined, as mentioned by its ‘DESCRIPTION’ parameter. Its parent node is ‘system’ and its own numerical identifier is 4. SNMP and PHP After having guided you through a brief introduction to SNMP and its management framework, it’s now time to talk about the functionality of the SNMP extension in PHP. The PHP extension has been part of PHP for a long time, and it was originally developed by Rasmus Lerdorf as the UCD-SNMP module. At that time it supported the cutting edge version of SNMP—version 1. Over time, minor modifications have been made to extension, mainly due to changes in the packages it depends on, and it is currently called NET-SNMP. Some 3 years ago, I wrote an SNMP extension that made it possible to take advantage of the security features that are part of SNMP version 3. This was never included in the main distribution for various reasons. However, ever since I returned to Europe and went to live in Italy, I started working on the extension again and have included the security features of SNMP version 3 in the main distribution for the next release of PHP. December 2002 · PHP Architect · www.phparch.com

SNMP Management With PHP As we have seen in the previous section, SNMP activities can really be divided in two kinds of operations: retrieving information on request from the manager’s side and initiating notifications from the agent’s. The PHP language only supports the first form. This is mostly for historical reasons, since the first versions of PHP were used as scripting tools for web-based systems in which a notification from the client would simply have been impossible to implement: PHP scripts in web pages are only executed upon a web client’s request, and this methodology obviously does not allow for a continuous listening for notifications. In addition, the probability that a notification be received exactly when a web page is requested is pretty much zero. Thus, the only SNMP-related commands available in PHP are management information-related. Generally they can be used in polling operations. PHP-SNMP functions The SNMP functions in PHP can be divided into two groups, helper functions and protocol functions. The protocol functions that we discuss here are those that exist in versions of PHP that have already been released, and are based on SNMPv1, as are the example scripts. However, the upcoming new release of PHP (4.3) will include the new SNMPv3 functions and will support the new protocol security features. These new functions are discussed later in this very article. Helper functions The helper functions (Figure 4) are used only to alter the format in which the values passed to and from the protocol functions are used. These functions are only for internal use and do not communicate to external SNMP agents. The only parameter that they accept is a boolean that indicates whether a particular feature is to be used or not. The example shown in Listing 1 illustrates a script in which the helper functions are used to change the output resulting from calls to the protocol functions. Their output is shown in Figure 5. Protocol functions The protocol functions (Figure 6), on the other hand, are used to communicate with the remote SNMP agents. They are all used for data retrieval, except for snmpset(), which can change management data in a remote SNMP agent. These functions almost all take the arguments shown in Figure 7 Listing 2 provides an example of how these functions can be used in a real-life scenario. I have commented the script in detail to make it easier for you to follow it. SNMP In Practice Now that we have introduced all of PHP’s SNMP

27

FEATURES

SNMP Management With PHP Listing 1

1

functionality, we’ll take a look at a few examples to show how scripts that allow polling of network management data from a remote SNMP agent can be written. For our first example, we want to create a script that provides generic system data about the remote server, such as its hostname (sysName), its location (sysLocation) and the person responsible (sysContact) for its administration. These are the managed objects that we have used earlier to explain the MIB tree and the MIB module syntax. Listing 3 contains two approaches that achieve this result in two different ways. The first method consists of a series of explicit calls to snmpget() to retrieve the required information from the remote SNMP agent. This results in 3 requests on the wire. The second approach, on the other hand, is based on an MIB branch retrieval and subsequent filtering of the information needed. This last method results in a high number of requests being made to the agent, and does not represent the optimal solution to the problem, since the entire information tree is retrieved, even though only three of its values are actually needed. The output of Listing 1 is shown in Figure 8. The Interface table The second example, shown in Listing 4, retrieves some columns from a table and demonstrates how Figure 4: Helper Functions bool snmp_get_quick_print (void) void snmp_set_quick_print (bool quick_print)

indexes are used. The table whose values are being outputted is the interface table, or “ifTable”, which is December 2002 · PHP Architect · www.phparch.com

defined in the IF-MIB module published in RFC 2863 (available at http://www.ietf.org/rfc/rfc2863.txt). This table maintains information about the interfaces in a system; an interface is defined as an entry point to the next underlying layer in a protocol stack. A common example of this is the TCP/IP stack in network interfaces and serial line connectors. The New SNMPv3 Functions Recently, I have added new functionality to the SNMP module so that it can support the SNMP version 3 specifications and, in particular, its security model. All the protocol functions available in the SNMPv1 module have an equivalent in SNMPv3. The only difference between the new functions and the old ones is that their ‘snmp’ prefix must be changed into ‘snmpv3’. Also, the community password string used for SNMPv1 must be replaced by the parameters shown in Figure 9. As you can see in Figure 10, upgrading current scripts to SNMPv3 is very easy—all you have to do is change the function names and introduce the new security parameters. The PHP team took this approach to encourage all developers to upgrade their scripts and deploy the new security features for the management of their systems. The next release of PHP will also contain two new SNMP helper functions shown in figure 11. I introduced them as solutions to problems I encountered trying to interpret object identifiers and enumeration values in my scripts. The first function, snmp_set_enum_print(), causes the module to manipulate the data produced by SNMP agents so that, if it is part of an enumeration, its “human readable” value will be returned rather than its numeric one. The second, snmp_set_numeric_print(), causes values

28

FEATURES

SNMP Management With PHP

Figure 5 sysName (quickprint default/’’):STRING:orville.cardano.lisanza.net sysName (quickprint ‘1’) :orville.cardano.lisanza.net sysName (quickprint ‘’) :STRING: orville.cardano.lisanza.netharrie@orville$

Listing 2 1


29

FEATURES

SNMP Management With PHP Figure 6: Syntax Protocol Functions

string snmpget (string target, string community, string object_id [, int timeout [, int retries]]) array snmprealwalk (string target, string community, string object_id [, int timeout [,int retries]]) bool snmpset (string target, string community, string object_id, string type, mixed value [, int timeout [, int retries]]) array snmpwalk (string target, string community, string object_id [, int timeout [, int retries]]) array snmpwalkoid (string target, string community, string object_id [, int timeout [, int retries]])

returned by a SNMP agent in numeric format not to be converted into human readable format. Numeric values are probably less intuitive to use for developers, but they possess the great advantage of remaining constant Figure 8 Explicit requests (SNMP-GET) The system name is: STRING: orville.cardano.lisanza.net The system administrator is: STRING: <[email protected]> The system location is: STRING: Cardano,Italy Tree retrieval, value selection (SNMP-WALK) The system name is: STRING: orville.cardano.lisanza.net The system administrator is: STRING: <[email protected]> The system location is: STRING: Cardano,Italy

and not being subject to change when moving from one system to another. Tips and tricks Many people dislike SNMP mainly because the do not understand how to read MIB modules or because they claim to have too little control over the SNMP

agent. Here are a couple of tricks that can make your life as a script developer a bit easier. 1) Determine first what you want to monitor from a high-level perspective and what is important for your deployment. This provides you as a developer with better guidance and direction and helps not to get lost in the SNMP details. 2) Use tools like libsmi (also open-source, and available from http://www.ibr.cs.tu-bs.de/projects/libsmi) that can help you by building an MIB tree for your viewing pleasure. If you use the ‘-f tree’ option, smidump will create a summary representation of the MIB modules that are part of a particular tree. This will enable you to determine the complete identifier of any object quickly and efficiently. 3) Use snmprealwalk() on your target agent as a tool to help you determine the identifiers of the objects you need. If you print out the results of a snmprealwalk() call, you can see what the human-readable form of a particular object identifier is. 4) Although this requires a bit more detailed knowledge of SNMP and MIB modules, use the numeric form of the object identifiers and the new snmp_set_numeric_print() function to retrieve the numeric form of your object identifiers. The advan-

Figure 7

string target

The target is used to define the hostname or the IP address of the remote SNMP agent as well to indicate the port (default 161) number the SNMP agent listens. Some examples are: “hostname” or “hostname:1661” where the first uses the default port number and the last the port number 1661 for the SNMP agent on the system ‘hostname’.

string community

The community string is an identification of the user/system that wants to access management information in an SNMP agent.

string object_id

The object identifier indicates the management information that is requested from the SNMP agent.

int timeout

The time out after which the SNMP request is resend if no reply is received.

int retries

The amount of retries that must be made before giving up.

string type and mixed value (only used for the snmpset)

The type and the value to which the object identifier in the remote SNMP agent will be set.


30

FEATURES


1

Figure 9 string username

The username identifies who wants access a remote SNMP agent. This could be compare to the old community string of SNMPv1/v2C. The security level defines the grade of security and only the following levels exist:

string sec_level

- ‘noAuthNoPriv’ or ‘nanp’; a security level that is similar to community-based security. This level does not require any of the authentication and privacy parameters to be set and therefore must be provided as “” empty strings. - 'authPriv' or 'ap'; the highest security level that provides authentication and encryption. This level does require both the authentication and the privacy parameters to be set.

string auth_protocol

The authentication protocol defines the sort of authentication used. The following protocols are supported 'MD5' and 'SHA'

string auth_passphrase

The authentication passphrase is used to generate an authentication key with which the authentication is done.

string priv_passphrase

The privacy protocol defines the encryption protocol and the following protocol is currently only supported ‘DES’.

string priv_passphrase

The privacy passphrase is a secret phrase to be used to generate an encryption key.


31

FEATURES


1

tage of using this approach is that numeric identifiers are unique in the MIB tree, and they never change— human readable forms may be modified. 5) Search for and read RFCs and books on SNMP. For instance, a good book with more in-depth analysis of the MIB modules is “Understanding SNMP MIBs” by D. Perkins and E. McGinnis, et al. (published by Prentice


Hall), while a good book for the SNMP protocol is “SNMPv1, SNMPv2c, SNMPv3 and RMON”, written by W. Stallings and published by Addison-Wesley. 6) Last but not least, don’t be frustrated if you don’t get everything working right away. Take your time and work through your problems one at a time.

32

FEATURES

SNMP Management With PHP Figure 10: Protocol Functions SNMPv3

string snmpv3get ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]]) array snmpv3realwalk ( string host, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]]) bool snmpv3set ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id, string type, mixed value [, int timeout [, int retries]]) array snmpv3walk ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]]) array snmpv3walkoid ( string hostname, string username, string seclevel, string auth_protocol, string auth_passphrase, string priv_protocol, string priv_passphrase, string object_id [, int timeout [, int retries]])

Signing Off Now that we have briefly introduced SNMP its related PHP functionality, I think that it’s fair to say that PHP is a simple and rich language for creating SNMP scripts. Due to the wide variety of extensions available for PHP, such as those that allow for database and graphics functionality, the potential is there to write complex management applications that provide very advanced capabilities. In fact, there are already two SNMP opensource packages available at www.sourceforge.net— Figure 11 void snmp_set_enum_print(bool enum_print) void snmp_set_numeric_print(bool numeric_print)

Just For Fun Network Management and PHP Network Analyzer—that make complex network analysis and management with PHP a reality. php|a

Harrie Hazewinkel is a long time participant in various open-source projects. He is the author of MOD-SNMP—an SNMP module with which the status of Apache can be monitored. He is an active participant in the Internet Engineering Task Force (IETF) and is the editor of the standard that defines managed objects for WWW services. His main expertise is Network Management and World Wide Web applications. Currently, he is an independent consultant and is open for new Internet challenges and adventures. You can reach him at [email protected].

www.cyberbite.com

CYBERBITE W W E E B B H H O O S S T T II N N G G Revolving around YOUR business Designed for PHP Programmers

Virtual Private Servers & Dedicated Servers

Reliable Internet hosting solutions


Guaranteed 99.95% uptime

33

FEATURES

FEATURES

A Concrete Look at Database Abstraction Layers By Maxim Maletsky What is all this Database Abstraction fever? Why do we make so much noise about it and create so many different solutions, none of which seem to be able to make everyone happy? Perhaps, because not everyone tried to understand what the actual goal for abstracting database access is.

E

ver since PHP came on the market, proclaiming itself to be the world’s most beloved programming language, there has been a lot of disappointment over its weak cross-database portability. Indeed, for many developers, supporting pretty much all the existing databases through individual modules was simply not enough. Developers also want the possibility to avoid hardcoding db-specific functions in their applications— they want to call only one module (or object), tell it what DBMS to use and access their data transparently. As it turns out, that wasn’t a reasonable addition to PHP—simply because of its nature. First of all, PHP is not a true object-oriented language—it uses procedural functions to interact with DBMS systems, and libraries for different databases have to be called in different ways. Second, DBMS extensions are, internally, independent of each other—it would thus be almost impossible to synchronize them into one single object without first creating an OOP wrapper, which brings us back to the initial problem. PHPLIB As PHP evolved, PHPLIB came into existence. PHPLIB was a PHP library whose main goal was to allow for session management in PHP3. Session information was saved in a database and, with the intent of providing cross-database functionality, a transparent layer was used for data access. Not only you could save sessions into your favorite DBMS, but there was also a way for


you to directly access these databases from your scripts, thus making PHPLIB the very first data abstraction layer available to PHP users. In PHPLIB, you’d call your database with something like this: This example would print an HTML table filled with 1

results. Handy, isn’t it? Before you could get away with accessing your database using just five lines of code, however, you’d have to specify what DBMS should have been used via an included file. That was convenient for most cases—particularly for projects where the DBMS could have changed at some point in the future. If that “lucky event” were to happen to you, you’d simply have to REQUIREMENTS PHP Version: 4.0 and up O/S: Any Web server: Any Support Tools: A Database System supported by PHP

34

FEATURES

A Concrete Look at Database Abstraction Layers

edit the main PHPLIB configuration file to point to a new database configuration file. Besides sessions and DB functionality, PHPLIB also provided you with lots of other goodies, such as an authentication system, HTML rendering classes and much more. Unfortunately for PHPLIB (and luckily for all of us), PHP4 came out with sessions incorporated by default, making them simpler to use and allowing for the full cross-platform compatibility with no need to use any database at all that everybody was waiting for. When that happened, PHPLIB started becoming less and less popular—to the point that it was used just for its DB abstraction capabilities and, of course, the other “goodies” that people were so addicted to. Alas, even that didn’t last for too long... PEAR Soon after PHP4 was released, the PEAR project began. PEAR is a library similar to PHPLIB, with the allimportant exception that it was developed officially along the PHP core itself. It contains an extremely powerful set of database abstraction functionality—as well as many more useful tools. There are dozens of people who are working on PEAR today and thousands of people who use it. One of the main goals of PEAR is to provide PHP with a flexible database access system available on any platform (and for any DBMS). An example of database use with PEAR is shown in Figure 1. The PEAR DB class makes it quite easy to work with multiple databases. PEAR also allows you to use Data Objects so you can build very dynamic queries and connect several of the other available PEAR components to it. For instance, if you wanted to create a query which selects all the employees who are between 18 and 35 years old, with Data Objects you could dynamically build it this way:

1

This would compose and internally execute the following query:

SELECT * FROM employees WHERE age > 12 AND age < 30


More DB Abstraction Modules Besides PHPLIB and PEAR, there are also a couple of other built-in extensions to PHP that allow for database abstraction, such as DBA and DBX. They are fast and flexible, but somewhat lacking in the compatibility department. There is also ADODB—a great and very powerful database library that is now available on a number of platforms. What the Ideal Database Abstraction Layer Should Do All these Database Abstraction Layers miss (at least) one thing. Sure, they do let you switch easily between all kinds of DBMS systems. The problem is that database systems do not simply differ in the way PHP interacts with them: they also often use a particular “dialect” of the SQL language that differs from the standard in subtle but significant ways. Thus, even though, theoretically, the word “Abstraction” between “Database” and “Layer” means you can accomplish the same task on any database server without editing a single line of your code, this is not entirely possible today because you will still have to modify your queries. For example, assume for a moment that we need to find out my age through SQL having only a string containing my birthday. In MySQL, the code will look like this: SELECT ( YEAR ( NOW() ) YEAR ( ‘1978-10-26 20:38:40’ ) ) as age;

In PostgreSQL, however, that will not work, and will have to be changed as follows:

SELECT ( EXTRACT( YEAR FROM TIMESTAMP ‘now’ ) EXTRACT( YEAR FROM TIMESTAMP ‘1978-10-26 20:38:40’ ) ) as age;

Oracle, on the other hand, requires quite a bit more work, as shown in Figure 2. All three SQL databases I used in the example claim to fully support SQL standards. The problem is, unfortunately, that the SQL standard is somewhat limited and not always adequate to the actual needs of every developer—as a result, you’ve just had quick proof that most DBMS systems do not (and, likely, never will) share the same SQL standard at all. Thus, if you’re used to having your SQL queries hardcoded in your applica-

35

FEATURES

A Concrete Look at Database Abstraction Layers

tion, you are going to have a very hard time rewriting most of them if you need to migrate it from one database to another. Ideally, a good Database Abstraction Layer should support differences in the SQL dialects for you. One way of doing this would be to dynamically rewrite the SQL code right within the db layer (such as ADODB does in some cases). However, as you can already imagine, this isn’t always a good solution, even if many DB layers try to implement it. For one thing, it’s not always possible to convert a SQL string written for a particular DBMS into its equivalent for a different DBMS in a univocal way; this means that it would be difficult (and dangerous) for the DB layer to “guess” what a SQL query means in every possible case. Another reason, and perhaps the most crucial one, is that parsing, “rethinking” and “repairing” every single SQL command would cause a significant performance hit that is not easily justified. The “good” database abstraction layers that we have today already slow things down enough—even without this feature. Finding a solution to this problem may look like an impossible dream. In reality, all it takes is a bit of foresight and some level of discipline. All you have to is to collect and save your SQL queries in a central location and use a sound dynamic mechanism to load them into

Theoretically, the word “Abstraction” between “Database” and “Layer” means you can accomplish the same task on any database server without editing a single line of your code. your code as needed. Consider the code in Listing 1—let’s call it mysql.sql.php—in which you hold your queries. As you can see, a separate variable is required for each SQL statement. Its value will later be injected into a call to a database, resulting in a system that can easily adapt to any DBMS—all you would have to do is to create another file, say postgresql.sql.php, that contains the SQL syntax appropriate for the DBMS you want to use. Of course, you will need a little framework that can use these files, providing you at the same time with the

rapX, don’t R.I.P. “Dearly Beloved, we are gathered here today to mark the passing of a fine young software product. So rich in promise, so beloved by its users, so carefully tended by its developers...”

rap-x is the collaboration platform that enables software developers to generate revenue from their existing users. Click for a demo at rap-X.com, or email [email protected] for more information


36

FEATURES

A Concrete Look at Database Abstraction Layers Listing 1: mysql.sql.php

1

php|architect (December 2002)

December

December

December

December

December)

December)

December

December

Discrete and Computational Geometry: Japanese Conference, JCDCG 2002, Tokyo, Japan, December 6-9, 2002, Revised Papers

High Performance Computing - HiPC 2002: 9th International Conference Bangalore, India, December 18-21, 2002, Proceedings

2002)

2002)

2002)

2002)

2002)

2002)

2002

2002)

2002)

2002)

2002)

2002)

2002)

2002)

2002)

2002

2002)

2002)

2002)

2002)

php|architect (December 2002)

December

December

December

December

December)

December)

December

December

Discrete and Computational Geometry: Japanese Conference, JCDCG 2002, Tokyo, Japan, December 6-9, 2002, Revised Papers

High Performance Computing - HiPC 2002: 9th International Conference Bangalore, India, December 18-21, 2002, Proceedings

2002)

2002)

2002)

2002)

2002)

2002)

2002

2002)

2002)

2002)

2002)

2002)

2002)

2002)

2002)

2002

2002)

2002)

2002)

2002)

Recommend Documents