Computer Vision Technology for Food Quality Evaluation

About the Editor Born in Southern China, Professor Da-Wen Sun is an internationally recognized figure for his leadersh...

Author: Da-Wen Sun

213 downloads 1902 Views 8MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

About the Editor

Born in Southern China, Professor Da-Wen Sun is an internationally recognized figure for his leadership in food engineering research and education. His main research activities include cooling, drying and refrigeration processes and systems, quality and safety of food products, bioprocess simulation and optimization, and computer vision technology. In particular, his innovative work on vacuum cooling of cooked meats, pizza quality inspection by computer vision, and edible films for shelf-life extension of fruit and vegetables have been widely reported in national and international media. Results of his work have been published in over 150 peer-reviewed journal papers and more than 200 conference papers. Dr Sun received First Class Honours BSc and MSc degrees in Mechanical Engineering and a PhD in Chemical Engineering in China before working in various universities in Europe. He became the first Chinese national to be permanently employed in an Irish University when he was appointed College Lecturer at National University of Ireland, Dublin (University College Dublin) in 1995, and was then continuously promoted in the shortest possible time to Senior Lecturer, Associate Professor and full Professor. Dr Sun is now a Professor and Director of the Food Refrigeration and Computerised Food Technology Research Group in the University College Dublin. As a leading educator in food engineering, Professor Sun has significantly contributed to the field of food engineering. He has trained many PhD students, who have made their own contributions to the industry and academia. Professor Sun has also given lectures on advances in food engineering on a regular basis to academic institutions internationally and delivered keynote speeches at international conferences. As a recognized authority in food engineering, he has been conferred adjunct/visiting/consulting professorships from ten top universities in China, including Zhejiang University, Shanghai Jiaotong University, Harbin Institute of Technology, China Agricultural University, South China University of Technology, Southern Yangtze University, etc. In

xii About the Editor

recognition of his significant contribution to food engineering worldwide and for his outstanding leadership in the field, the International Commission of Agricultural Engineering (CIGR) awarded him the CIGR Merit Award in 2000 and again in 2006; the Institution of Mechanical Engineers (IMechE) based in the UK named him “Food Engineer of the Year 2004.” Professor Sun is a Fellow of the Institution of Agricultural Engineers. He has also received numerous awards for teaching and research excellence, including the President’s Research Fellowship, and has twice received the President’s Research Award of University College Dublin. He is a member of the CIGR Executive Board and Honorary Vice-President of CIGR, the editor-in-chief of Food and Bioprocess Technology – an International Journal (Springer), the former editor of the Journal of Food Engineering (Elsevier), the series editor of the “Contemporary Food Engineering” book series (CRC Press/Taylor & Francis), and an editorial board member for the Journal of Food Process Engineering (Blackwell), Sensing and Instrumentation for Food Quality and Safety (Springer), and the Czech Journal of Food Sciences. He is also a Chartered Engineer registered in the UK Engineering Council.

Contributors

Mohd. Zaid Abdullah (Chs 1, 20), School of Electrical and Electronic Engineering, Engineering Campus, Universiti Sains Malaysia, 14300 Penang, Malaysia Murat O. Balaban (Ch. 8), University of Florida, Food Science and Human Nutrition Department, PO Box 110370, Gainesville, FL 32611-0370, USA Jose Blasco (Ch. 10), IVIA (Instituto Valenciano de Investigaciones Agrarias), Cra. Moncada-Naquera km 5, 46113 Moncada (Valencia), Spain Sibel Damar (Ch. 8), University of Florida, Food Science and Human Nutrition Department, PO Box 110370, Gainesville, FL 32611-0370, USA Ricardo Díaz (Ch. 12), Instrumentation and Automation Department, Food Technological Institute AINA, Paterna (Valencia) 46980, Spain Cheng-Jin Du (Chs 4, 6, 18), Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland Prabal K. Ghosh (Ch. 15), Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB, Canada, R3T 5V6 Sundaram Gunasekaran (Ch. 19), Food and Bioprocess Engineering Laboratory, University of Wisconsin-Madison, Madison, WI 53706, USA Dave W. Hatcher (Ch. 21), Wheat Enzymes & Asian Products, Canadian Grain Commission, Winnipeg, MB, Canada, R3C 3G8 Digvir S. Jayas (Ch. 15), Stored-Grain Ecosystems, Winnipeg, MB, Canada, R3T 2N2 Chithra Karunakaran (Ch. 15), Canadian Light Source, Saskatoon, Saskatchewan, Canada, S7N 0X4 Olivier Kleynen (Ch. 9), Unité de Mécanique et Construction, Gembloux Agricultural University, Passage des Déportés, 2, B-5030 Gembloux, Belgium Vincent Leemans (Ch. 9), Unité de Mécanique et Construction, Gembloux Agricultural University, Passage des Déportés, 2, B-5030 Gembloux, Belgium Renfu Lu (Ch. 14), US Department of Agriculture, Agricultural Research Service, Sugar beet and Bean Research Unit, Michigan State University, East Lansing, MI 48824, USA Thierry Marique (Chs 13, 22), Centre Agronomique de Researches Appliquees du Hainaut (CARAH), 7800 Ath, Belgium Domingo Mery (Chs 13, 22), Departamento de Ciencia de la Computacion, Pontificia Universidad Católica de Chile, Av. Vicuña Mackenna 4860 (143), Santiago, Chile

xiv Contributors

Enrique Moltó (Ch. 10), IVIA (Instituto Valenciano de Investigaciones Agrarias), Cra. Moncada-Naquera km 5, 46113 Moncada (Valencia), Spain Masateru Nagata (Ch. 11), Faculty of Agriculture, University of Miyazaki, Miyazaki, 889-2192 Japan Asli Z. Odaba¸si (Ch. 8), University of Florida, Food Science and Human Nutrition Department, PO Box 110370, Gainesville, FL 32611-0370, USA Yukiharu Ogawa (Ch. 16), Faculty of Horticulture, Chiba University, Matsudo, Chiba, 271-8510 Japan Alexandra C.M. Oliveira (Ch. 8), Fishery Industrial Technology Center, University of Alaska, Fairbanks, Kodiak, AK 99615, USA Jitendra Paliwal (Ch. 15), Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada Bosoon Park (Ch. 7), US Department of Agriculture, Agricultural Research Service, Richard B. Russell Research Center, Athens, GA 30605, USA Franco Pedreschi (Chs 13, 22), Universidad de Santiago Chile, Departamento de Ciencia y Tecnologia de Alimentos, Facultad Tecnologica, Av. Ecuador 3769, Santiago, Chile Ricardo Díaz Pujol (Ch. 12), Dpto Instrumentación y Automática AINIA – Instituto Tecnológico Agroalimentario, 46980 Paterna, Valencia, Spain Muhammad A. Shahin (Ch. 17), Grain Research Laboratory, Canadian Grain Commission, Winnipeg, MB, Canada, R3C 3G8 Da-Wen Sun (Chs 2, 3, 4, 5, 6, 18), Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland Stephen J. Symons (Ch. 17), Grain Research Laboratory, Canadian Grain Commission, Winnipeg, MB, Canada, R3C 3G8 Jasper G. Tallada (Ch. 11), Faculty of Agriculture, University of Miyazaki, United Graduate School of Agricultural Sciences, Kagoshima University, Miyazaki, 889-2192 Japan Jinglu Tan (Ch. 5), Department of Biological Engineering, University of Missouri, Columbia, MO 65211, USA Chaoxin Zheng (Chs 2, 3), Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland Liyun Zheng (Ch. 5), Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland

Preface

Based on image processing and analysis, computer vision is a novel technology for recognizing objects and extracting quantitative information from digital images in order to provide objective, rapid, non-contact, and non-destructive quality evaluation. Driven by significant increases in computer power and rapid developments in imageprocessing techniques and software, the application of computer vision has been extended to the quality evaluation of diverse and processed foods. In recent years in particular, computer vision has attracted much research and development attention; as a result, rapid scientific and technological advances have increasingly taken place regarding the quality inspection, classification, and evaluation of a wide range of food and agricultural products. As the first book in this area, Computer Vision Technology for Food Quality Evaluation focuses on these recent advances. The book is divided into five parts. Part I provides an outline of the fundamentals of the technology, addressing the principles and techniques for image acquisition, segmentation, description, and recognition. Part II presents extensive coverage of the application in the most researched areas of fresh and cooked meats, poultry, and seafood. Part III details the application of computer vision in the quality evaluation of agricultural products, including apples, citrus, strawberry, table olives, and potatoes. Using computer vision to evaluate and classify the quality of grains such as wheat, rice and corn is then discussed in Part IV. The book concludes with Part V, which is about applying computer vision technology to other food products, including pizza, cheese, bakery, noodles, and potato chips. Computer Vision Technology for Food Quality Evaluation is written by international peers who have both academic and professional credentials, with each chapter addressing in detail one aspect of the relevant technology, thus highlighting the truly international nature of the work. The book therefore provides the engineer and technologist working in research, development, and operations in the food industry with critical, comprehensive, and readily accessible information on the art and science of computer vision technology. It should also serve as an essential reference source for undergraduate and postgraduate students and researchers in universities and research institutions.

Image Acquisition Systems Mohd. Zaid Abdullah School of Electrical and Electronic Engineering, Engineering Campus, Universiti Sains Malaysia, 14300 Penang, Malaysia

1 Introduction In making physical assessments of agricultural materials and foodstuffs, images are undoubtedly the preferred method in representing concepts to the human brain. Many of the quality factors affecting foodstuffs can be determined by visual inspection and image analysis. Such inspections determine market price and, to some extent, the “best-used-before” date. Traditionally, quality inspection is performed by trained human inspectors, who approach the problem of quality assessment in two ways: seeing and feeling. In addition to being costly, this method is highly variable and decisions are not always consistent between inspectors or from day to day. This is, however, changing with the advent of electronic imaging systems and with the rapid decline in cost of computers, peripherals, and other digital devices. Moreover, the inspection of foodstuffs for various quality factors is a very repetitive task which is also very subjective in nature. In this type of environment, machine vision systems are ideally suited for routine inspection and quality assurance tasks. Backed by powerful artificial intelligence systems and state-of-the-art electronic technologies, machine vision provides a mechanism in which the human thinking process is simulated artificially. To date, machine vision has extensively been applied to solve various food engineering problems, ranging from simple quality evaluation of food products to complicated robot guidance applications (Tao et al., 1995; Pearson, 1996; Abdullah et al., 2000). Despite the general utility of machine vision images as a first-line inspection tool, their capabilities regarding more in-depth investigation are fundamentally limited. This is due to the fact that images produced by vision cameras are formed using a narrow band of radiation, extending from 10−4 m to 10−7 m in wavelength. For this reason, scientists and engineers have invented camera systems that allow patterns of energy from virtually any part of the electromagnetic spectrum to be visualized. Camera systems such as computed tomography (CT), magnetic resonance imaging (MRI), nuclear magnetic resonance (NMR), single photon emission computed tomography (SPECT) Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0

Copyright © 2008, Elsevier Inc. All rights reserved

1

4 Image Acquisition Systems

and positron emission tomography (PET) operate at shorter wavelengths ranging from 10−8 m to 10−13 m. Towards the opposite end of the electromagnetic spectrum there are infrared and radio cameras, which enable visualization to be performed at wavelengths greater than 10−6 m and 10−4 m, respectively. All these imaging modalities rely on acquisition hardware featuring an array or ring of detectors which measure the strength of some form of radiation, either following reflection or after the signal has passed transversely through the object. Perhaps one thing that these camera systems have in common is the requirement to perform digital image processing of the resulting signals using modern computing power. Whilst digital image processing is usually assumed to be the process of converting radiant energy in a three-dimensional world into a two-dimensional radiant array of numbers, this is certainly not so when the detected energy is outside the visible part of the spectrum. The reason is that the technology used to acquire the imaging signals is quite different, depending on the camera modalities. The aim of this chapter is therefore to give a brief review of the present state-of-the-art image acquisition technologies that have found many applications in the food industry. Section 2 summarizes the electromagnetic spectrum which is useful in image formation. Section 3 describes the principles of operation of machine vision technology, along with illumination and electronics requirements. Other imaging modalities, particularly the acquisition technologies operating at the non-visible range, are briefly discussed in Section 4. In particular, technologies based on ultrasound, infrared, MRI and CT are addressed, followed by some of their successful applications in food engineering found in the literature. Section 5 concludes by addressing likely future developments in this exciting field of electronic imaging.

2 The electromagnetic spectrum As discussed above, images are derived from electromagnetic radiation in both visible and non-visible ranges. Radiation energy travels in space at the speed of light in the form of sinusoidal waves with known wavelengths. Arranged from shorter to longer wavelengths, the electromagnetic spectrum provides information on the frequency as well as the energy distribution of the electromagnetic radiation. Figure 1.1 shows the electromagnetic spectrum of all electromagnetic waves. Referring to Figure 1.1, the gamma rays with wavelengths of less than 0.1 nm constitute the shortest wavelengths of the electromagnetic spectrum. Traditionally, gamma radiation is important for medical and astronomical imaging, leading to the development of various types of anatomical imaging modalities such as CT, MRI, SPECT, and PET. In CT the radiation is projected onto the target from a diametrically opposed source, whilst with others it originates from the target – by simulated emission in the case of MRI, and through the use of radiopharmaceuticals in SPECT and PET. At the other end of the spectrum, the longest waves are radio waves, which have wavelengths of many kilometers. The well-known ground-probing radar (GPR) and other microwave-based imaging modalities operate in this frequency range.

The electromagnetic spectrum 5

0.4 UV

0.5 BLUE

0.6 GREEN

0.7 RED

IR

Visible

Wavelength (␮m) 10⫺6

10⫺5

10⫺4

Wavelength (␮m) 10⫺3

X-rays Gamma rays

10⫺2

10⫺1

Ultraviolet (UV)

1

10

1

102

103

104

105

106

107

108

Infrared (IR) Microwaves Increasing resolution Increasing energy

Radio waves

Decreasing wavelength

Figure 1.1 The electromagnetic spectrum comprising the visible and non-visible range.

Located in the middle of the electromagnetic spectrum is the visible range, consisting of narrow portion of the spectrum with wavelengths ranging from 400 nm (blue) to 700 nm (red). The popular charge-coupled device or CCD camera operates in this range. Infrared (IR) light lies between the visible and microwave portions of the electromagnetic band. As with visible light, infrared has wavelengths that range from near (shorter) infrared to far (longer) infrared. The latter belongs to the thermally sensitive region, which makes it useful in imaging applications that rely on the heat signature. One example of such an imaging device is the indium gallium arsenide (InGaAs)-based near-infrared (NIR) camera, which gives the optimum response in the 900–1700-nm band (Deobelin, 1996). Ultraviolet (UV) light is of shorter wavelength than visible light. Similar to IR, the UV part of the spectrum can be divided, this time into three regions: near ultraviolet (NUV) (300 nm) (NUV), far ultraviolet (FUV) (30 nm), and extreme ultraviolet (EUV) (3 nm). NUV is closest to the visible band, while EUV is closest to the X-ray region and therefore is the most energetic of the three types. FUV, meanwhile, lies between the near and extreme ultraviolet regions, and is the least explored of the three. To date there are many types of CCD camera that provide sensitivity at the near-UV wavelength range. The sensitivity of such a camera usually peaks at around 369 nm while offering coverage down to 300 nm.


Mathematically, the wavelength (λ), the frequency ( f ), and the energy (E) are related by Planck’s equation: c (1.1) E=h λ where h is the Planck’s constant (6.626076 × 10−34 J s), and c is the speed of light (2.998 × 10−34 m/s). Consequently, the energy increases as the wavelength decreases. Therefore, gamma rays, which have the shortest wavelengths, have the highest energy of all the electromagnetic waves. This explains why gamma rays can easily travel through most objects without being affected. In contrast, radio waves have the longest wavelength and hence the lowest energy. Therefore, their penetrative power is at least hundreds order of magnitude lower than that of gamma or X-rays. Moreover, both gamma and X-rays travel in a straight line and their paths are not affected by the object through which these signals propagate. This is known as the hard-field effect. Conversely, radiowaves do not travel in straight lines and their paths depend strongly on the medium of propagation. This is the soft-field effect. Both the hard- and softfield effects have a direct effect on the quality of images produced by these signals. The soft-field effect causes many undesirable artefacts, most notably, image blurring, and therefore images produced by gamma rays generally appear much clearer than do images produced by radiowaves. Another important attribute that is wavelengthdependent is image resolution. In theory, the image spatial resolution is essentially limited to half of the interrogating wavelength, and therefore the spatial resolution also increases as the wavelength decreases. Thus, the resolution of typical gamma rays is less than 0.05 nm, enabling this type of electromagnetic wave to “see” extremely small objects such as water molecules. In summary, these attributes, along with the physical properties of the sensor materials, establish the fundamental limits to the capability of imaging modalities and their applications. The following sections explain the technology of image acquisition and applications for all the imaging modalities discussed, focusing on the visible modality or computer vision system, since this device has extensively been used for solving various food engineering problems. Moreover, given the progress in computer technology, computer vision hardware is now relatively inexpensive and easy to use. To date, some personal computers offer capability for a basic vision system by including a camera and its interface within the system. However, there are specialized systems for vision, offering performance in more than one aspect. Naturally, as with any specialized equipment, such systems can be expensive.

3 Image acquisition systems In general, images are formed by incident light in the visible spectrum falling on a partially reflective, partially absorptive surface, with the scattered photons being gathered up in the camera lens and converted to electrical signals either by vacuum tube or by CCD. In practice, this is only one of many ways in which images can be generated. Generally, thermal and ultrasonic methods, X-rays, radiowaves, and other techniques can all generate an image. This section examines the methods and procedures by which images are generated for computer vision applications, including tomography.

Image acquisition systems 7

3.1 Computer vision The hardware configuration of computer-based machine vision systems is relatively standard. Typically, a vision system consists of: • • • •

•

an illumination device, which illuminates the sample under test a solid-state CCD array camera, to acquire an image a frame-grabber, to perform the A/D (analog-to-digital) conversion of scan lines into picture elements or pixels digitized in a N row by M column image a personal computer or microprocessor system, to provide disk storage of images and computational capability with vendor-supplied software and specific application programs a high-resolution color monitor, which aids in visualizing images and the effects of various image analysis routines.

Figure 1.2 shows a typical set-up, such as an investigator needs to start experimenting with machine vision applications. All essential components are commercially available, and the price for the elementary system can be as low as £2000.00. The set-up shown in Figure 1.2 is an example of a computer vision system that can be found in many food laboratories, mainly for research and imaging applications. In this case, the objective is ultimately to free human inspectors from undertaking tedious, laborious, time-consuming, and repetitive inspection tasks, allowing them to focus on more demanding and skilled jobs. Computer vision technology not only provides a high level of flexibility and repeatability at a relatively low cost, but also, and more importantly, it permits fairly high plant throughput without compromising accuracy. The food industry continues to be among the fastest-growing segments of machine vision application, and it ranks among the top ten industries that use machine vision systems (Gunasekaran, 1996). Currently, several commercial vendors offer automatic vision-based quality evaluation for the food industry. Even though machine vision systems have become increasingly simple to use, the applications themselves can still be extremely complicated. A developer needs to know precisely what must be achieved in order to ensure successful implementation

CCD color camera

Test station

Illumination system

BNC cable

Sample under test Color framegrabber

Figure 1.2 Essential elements of a typical computer vision system.


of a machine vision application. Key characteristics include not only the specific part dimensions and part tolerances, but also the level of measurement precision required and the speed of the production line. Virtually all manufacturing processes will produce some degree of variability and, while the best machine vision technology is robust enough to compensate automatically for minor differences over time, the applications themselves need to take major changes into account. Additional complexity arises for companies with complex lighting and optical strategies, or unusual materials-handling logistics. For these reasons, it is essential to understand the characteristics of the part and sub-assemblies of the machine system, as well as the specifications of the production line itself. 3.1.1 Illumination

The provision of correct and high-quality illumination, in many vision applications, is absolutely decisive. Despite the advances of machine vision hardware and electronics, lighting for machine vision remains the art for those involved in vision integration. Engineers and machine vision practitioners have long recognized lighting as being an important piece of the machine vision system. However, choosing the right lighting strategy remains a difficult problem because there is no specific guideline for integrating lighting into machine vision applications. In spite of this, some rules of thumb exist. In general, three areas of knowledge are required to ensure a successful level of lighting for the vision task: 1. Understanding of the role of the lighting component in machine vision applications 2. Knowledge of the behavior of light on a given surface 3. Understanding of the basic lighting techniques available that will allow the light to create the desired feature extraction. In the vast majority of machine vision applications, image acquisition deals with reflected light, even though the use of backlit techniques can still be found. Therefore, the most important aspect of lighting is to understand what happens when light hits the surface – more specifically, to know how to control the reflection so that the image appears of a reasonably good quality. Another major area of concern is the choice of illuminant, as this is instrumental in the capability of any form of machine vision to represent the image accurately. This is due to the fact that the sensor response of a standard imaging device is given by a spectral integration process (Matas et al., 1995). Mathematically, λ2 pxk

=

ρk (λ)L(λ)dλ

(1.2)

λ1

where pxk is the response of the kth sensor at location x of the sensor array, ρk (λ) is the responsitivity function of the kth sensor, and L(λ) is the light reflected from the surface that is projected on pixel x. For a CCD camera the stimulus L(λ) is the product of the spectral power distribution S(λ) of the light that illuminates the object, and the


spectral reflectance C(λ) of the camera itself, i.e. L(λ) = S(λ)C(λ)

(1.3)

Hence, two different illuminants, S1 (λ) and S2 (λ), may yield different stimuli using the same camera. Therefore, the illuminant is an important factor that must be taken into account when considering machine vision integration. Frequently, knowledgeable selection of an illuminant is necessary for specific vision applications. Traditionally, the two most common illuminants are fluorescent and incandescent bulbs, even though other light sources (such as light-emitting diodes (LEDs) and electroluminescent sources) are also useful. Figure 1.3 shows the spectral distributions of three different light sources: the sun, an incandescent bulb, and standard cool white fluorescent light. Referring to Figure 1.3, the only difference between daylight and electric light is the amount of energy emitted at each wavelength. Even though the light energy itself is fundamentally the same, however, the optimum light will have more intensity than the other sources. When the light is not as intense as it should be, three possible damaging effects occur: 1. There may not be sufficient signal-to-noise ratio at the camera 2. The electrical noise tends to increase as the light gets dimmer and less intense 3. Most importantly, a less intense light will cause a significant loss in the camera depth-of-field. Additionally, effects from ambient light are more likely to occur under poor lighting conditions. Referring to Figure 1.3 again, it can be seen that the incandescent source has a fairly normal distribution over the visible spectrum while the fluorescent source has sharp

1.0 0.9

Nomalized spectral power

0.8

Daylight

0.7 0.6 0.5 0.4

Incandescent

0.3 0.2 Cool white fluorescent

0.1 0 350

400

450

500

550

600

650

700

750

800

Wavelength (nm) Figure 1.3 Comparison in relative spectral energy distribution between daylight, incandescent, and cool white fluorescent light (Stiles and Wyszecki, 2000).


peaks in some regions. This means that objects under an incandescent source produce an image with a much lower signal-to-noise ratio. This is not acceptable in some cases, especially those that are concerned with color-image processing (Daley et al., 1993). In contrast, fluorescent bulbs are inherently more efficient, and produce more intense illumination at specific wavelengths. Moreover, fluorescent light provides a more uniform dispersion of light from the emitting surface, and hence does not require the use of diffusing optics to disseminate the light source over the field of view, as is the case with incandescent bulbs. For these reasons, a fluorescent bulb, particularly the cool white type, is a popular choice for many machine vision practitioners (Tao et al., 1995; Abdullah et al., 2001, 2005; Pedreschi et al., 2006). However, care must be taken when using the fluorescent light, as this source is normally AC driven. The 50-Hz fluorescent bulb usually introduces artefacts in the image resulting from the oversampling of the analog-to-digital converter. In order to reduce flickering, highfrequency fluorescent bulbs, operating at a frequency in the range of a few tens of kilohertz, are preferred rather than low-frequency ones. Apart from the illuminant, the surface geometry is also important in the illumination design. The key factor is to determine whether the surface is specular or diffuse. Light striking a diffuse surface is scattered because of the multitude of surface angles. In comparison, light striking a glossy surface is reflected at the angle of incidence. Therefore, the position of an illuminant is very important in order to achieve high contrast in an image. There are two common geometries for the illuminators: the ring illuminator and the diffuse illuminator (see Figure 1.4). The ring illuminator has the simplest geometry and is generally intended for general purpose applications, especially for imaging flat surfaces. The diffuse illuminator, meanwhile, delivers virtually 180◦ of diffuse illumination, and is used for imaging challenging reflective objects. Since most food products are basically 3D objects, the diffuse illuminator is well suited for this imaging application. However, there has been some success in using the ring illuminator to solve lighting problems in food engineering. For instance, a ring illuminator together with a 90-kHz ultra highfrequency fluorescent bulb has been found to be effective in the color- and shapegrading of star fruits (Abdullah et al., 2005). In an attempt to produce uniform lighting, Paulsen (1990) mounted a ring light in a cylindrically-shaped diffuse lighting chamber.

Camera

Camera Light source

Light source Object (a) Figure 1.4

Light source Object (b)

Two possible lighting geometries: (a) the ring illuminator; (b) the diffuse illuminator.


Such a set-up is extremely useful for visual inspection of grains and oilseed, with the success rate reaching almost 100 percent. In spite of the general utility of the ring illuminator, however, the majority of machine vision applications are based on the diffuse illuminator. Heinemann et al. (1994) employed this type of illumination system for the shape-grading of mushrooms. The same system was investigated by Steinmetz et al. (1996) in the quality grading of melons. Both groups of authors have reported successful application of machine vision, with a grading accuracy that exceeds 95 percent. There are many other applications involving diffuse illuminator and computer vision integration. Batchelor (1985) reviewed some of the important factors to be considered when designing a good illumination system. 3.1.2 Electronics

Capturing the image electronically is the first step in digital image processing. Two key elements are responsible for this: the camera and the frame-grabber. The camera converts photons to electrical signals, and the frame-grabber then digitizes these signals to give a stream of data or bitmap image. There are many types of camera, ranging from the older pick-up tubes such as the vidicons to the most recent solid-state imaging devices, such as the Complementary Metal Oxide Silicon (CMOS) cameras. The latter is the dominant technology for cameras, and revolutionized the science of imaging with the invention of the CCD device in 1970. As CCD cameras have less noise, higher sensitivity and a greater dynamic range, they have also become the device of choice for a wide variety of food engineering applications. In general, the CCD sensor comprises a photosensitive diode and a capacitor connected in parallel. There are two different modes in which the sensor can be operated: passive and active. Figure 1.5 shows the details of the schematics. Referring to Figure 1.5, the photodiode converts light into electrical charges, which are then stored in the capacitor. The charges are proportional to the light intensity. In passive mode, these charges are transferred to a bus line when the “select” signal is activated. In the active mode, charges are first amplified before being transferred to a bus line, thus compensating the limited fill factor of the photodiode. An additional “reset” signal allows the capacitor to be discharged when an image is rescanned.

Select

Select

Reset FET transistors

FET transistor Column bus

Column bus

Light Capacitor

Photodiode

VDD

Light

Capacitor

(a) Figure 1.5 Sensor operation in (a) passive mode and (b) active mode.

Photodiode

(b)


CCD elements

CCD cells …

Integration area

Shift register (a) …

…

Shift register

…

…

Shift register

…

Shift register

…

Shift register

…

…

Storage area

… …

Shift register (b) Figure 1.6

Shift register (c)

Three possible CCD architectures: (a) linear, (b) interline, and (c) frame-transfer.

Depending on the sensing applications, CCD imagers come in various designs. The simplest form is the linear CCD scanner, which is shown schematically in Figure 1.6a. This design is used mostly in office scanner machines. It consists of a single row of photodiodes, which capture the photons. The sensors are lined up adjacent to a CCD shift register, which does the readout. The picture or document to be scanned is moved, one line at a time, across the scanner by mechanical or optical means. Figures 1.6b and 1.6c show two-dimensional CCD area arrays, which are mostly associated with modern digital cameras. The circuit in Fig. 1.6b portrays the interline CCD architecture, while Figure 1.6c shows that of a frame-transfer imager. Basically, the interline CCD comprises of a stack of vertical linear scanners connected by an additional, horizontal shift register that collects and passes on the charge readout from linear scanners, row by row. In the case of frame-transfer architecture, the CCD elements, the entire surfaces of which are covered by photosensitive devices, form the photo-sensing area. It can be seen from Figure 1.6c that the frame-transfer design comprises integration and storage areas, forming the integration and storage frames, respectively. The storage-frame array captures an image and transfers the charge to the adjacent storage-frame array. In this way, the integration array can capture a new image while the storage array reads the previous image. Both interline and frame-transfer architectures are suitable for capturing motion images, whilst the linear scanner is best suited for scanning still pictures. Full-frame CCD cameras with four million pixels


Video signal

Programmable acquisition

A/D converter

Host 32-bit PCI bus

Look-up table

Image buffer

PCI bus interface

Image-buffer control

Figure 1.7 General structure of a frame-grabber card, showing some important elements.

and a frame rate of more than 30 frames per second (fps) are now commercially available. Modern CCD cameras come with analog, or digital, or both outputs. The analog signals are conform with the European CCIR (Comite Consultatif International des Radiocommunication) or US RS170 video standards. In spite of a reduced dynamic range, analog cameras work well for slower applications ( g1 (x, y) > f1 (x, y). The interactions between ultrasound and the sample can be explained as follows. Probe A transmits the ultrasonic wave, which travels in a straight line until it reaches the f1 (x, y) and g1 (x, y) interface, which causes reflection. This is detected by the same probe, which now acts as a receiver. The amplified signals are fed into the y-plates of the oscilloscope, and a timebase is provided, synchronized to the transmitter pulse. Some of the energy, however, continues to travel until it reaches the f1 (x, y) and g2 (x, y) interface, where some energy is again reflected and hence detected by A. In similar fashion, some of the remaining energy continues to travel until it reaches probe B, where it is again detected and measured. Consequently, probe A provides a graph, detailing the echo signal, in which the height corresponds to the size of the inhomegeneity and the timebase provides its range or depth. Such a pattern is known as an A-scan (see Figure 1.8b). Pulse-shaper Amplitude

Amplifier

Filter/rectifier Amplifier Generator

0

t1

Time

t2 (b)

B Rx

g2(x,y) g1(x,y)

A Rx/Tx

Amplitude

f1(x,y)

Sample

Oscilloscope

0 (a)

(c)

t1

Figure 1.8 Ultrasonic measuring system showing (a) essential elements, (b) reflection, and (c) transmission measurements.

Time


Figure 1.8c shows the attenuated transmitted energy as observed by probe B. Both graphs show that information relating to the amplitude of both the transmitted and the reflected pulses can be measured, and this can also be used for imaging. As shown in Figure 1.8, the signals are usually rectified and filtered to present a simple one-dimensional picture, and the timebase can be delayed to allow for a couplant gap. To provide a full two-dimensional image, the ultrasonic probe must be moved over the surface of the sample. The Tx/Rx probe is connected via mechanical linkages to position transducers, which measure its x and y coordinates and its orientation. In this case, the output signals determine the origin and direction of the probe, while the amplitude of the echo determines the spot brightness. As the probe is rotated and moved over the sample, an image is built and retained in a digital store. This procedure is known as a B-scan, and produces a “slice” through a sample, normal to the surface. In contrast, a C-scan produces an image of a “slice” parallel to the surface. In order to produce the C-scan image, the ultrasonic probe must again be moved but this time over the volume of the sample. The time, together with the x and y coordinates of the image displayed, represents the lateral movement of the beam across the plane. By time-gating the echo signals, only those from the chosen depth are allowed to brighten the image. C-scan images may be produced using the same equipment as for B-scanning. Most of the studies in the literature rely on the use of A-scan or B-scan methods, probably because the C-scan image does not provide any additional information which is useful for further characterization. Regardless of the methods, the ultrasound images generally share at least three common drawbacks: 1. Low image spatial resolution – typically of a few millimeters 2. Low signal-to-noise ratio 3. Many artefacts. The first of these is related to the wavelength and hence frequency of ultrasound, which typically ranges from 2 to 10 MHz. In order to improve the resolution, some ultrasound devices operate at frequencies higher than this; however, such devices must be used with care because the skin effect increases with increasing frequency. The factors therefore have to be balanced against each other. The second and third drawbacks are due to the coherent nature of the sound wave and the physics of reflection. Any coherent pulse will interfere with its reflected, refracted, and transmitted components, giving rise to speckle, similar to the speckle observed in laser light (Fishbane et al., 1996). On the other hand, reflection occurs when the surface has a normal component parallel to the direction of the incident wave. Interfaces between materials that are parallel to the wave will not reflect the wave, and are therefore not seen in ultrasound images; such parallel interfaces form a “hole” in the ultrasound image. Despite these drawbacks, the technique is safe and relatively inexpensive. Current research methods tend to eliminate artefacts, improve image contrast, and simplify the presentation of data, and many efforts are being directed towards three-dimensional data acquisition and image representation.


3.3 Infrared When both computer vision and ultrasound systems fail to produce the desired images, food engineers and technologists could presumably resort to the use of much longer wavelengths for image acquisition. In the region of 700–1000 nm lies the infrared (IR) range, and the technique responsible for generating images with infrared light is known as thermographic photography. Thermographic imaging is based on the simple fact that all objects emit a certain amount of thermal radiation as a function of their temperature. Generally, the higher the temperature of the object, the more IR radiation it emits. A specially built camera, known as an IR camera, can detect this radiation in a way similar to that employed in an ordinary camera for visible light. However, unlike computer vision, thermal imaging does not require an illumination source for spectral reflectance, which can be affected by the varied surface color of a target or by the illumination set-up. Thermographic signatures of food are very different for different materials, and hence IR imaging has found applications and many other uses in the food industry – such as identification of foreign bodies in food products (Ginesu et al., 2004). Moreover, many major physiological properties of foodstuffs (firmness, soluble-solid content, and acidity) appear to be highly correlated with IR signals, implying that image analysis of IR thermography is suitable for quality evaluation and shelf-life determination of a number of fruit and vegetable products (Gómez et al., 2005). Therefore, thermal imaging offers a potential alternative technology for non-destructive and non-contact image-sensing applications. Good thermographic images can be obtained by leaving the object at rest below the IR camera, applying a heat pulse produced by a flashlight, and monitoring the decreasing temperature as a function of time. Because of different thermal capacities or heat conductivities, the objects will cool down at different speeds; therefore, the thermal conductivity of an object can be measured by the decreasing temperature calculated from a sequence of IR images. Using these relatively straightforward procedures, Ginesu et al. (2004) performed experiments on objects with different thermal properties, aiming to simulate foreign-body contamination in real experiments. Both the long (500-fps) and short (80-fps) sequence modes were used to record the images, enabling the radiation patterns of objects with low and high thermal capacities, respectively, to be monitored. Temperature data were presented in terms of average gray levels computed from 10 × 10 image pixels in the neighborhood of each object. Figure 1.9 shows the results. It can be seen from Figure 1.9 that the cardboard and the wooden stick behave quite differently from other materials, as they appear to be much hotter at the beginning but decrease in temperature rather quickly. This is due to the fact that these materials are dry and light, whereas foods contain a large quantity of water, which heats up more slowly and reaches lower temperatures, thus maintaining the heat for a longer time and cooling down slowly. By plotting and analyzing the absolute differences between radiation curves of different materials, it is possible to distinguish between food and foreign objects. Theoretically, the existence of such unique thermal signatures of different materials is due to the concept of a black body, defined as an object that does not reflect


64800 Stone Cardboard Metal chip Wooden stick Almond Raisin Nut

Average gray value

63800

62800

61800

60800

59800

58800

0

1.1

2.2

3.3

4.4

5.5

Time (s) Figure 1.9 Decreasing temperature curves of different materials plotted as a function of time (Ginesu et al., 2004).

any radiation. Planck’s law describes the radiation emission from a black body as (Gaossorgues, 1994): R(λ, θ) =

2πhc 2 λ−5 hc exp −1 λσθ

(1.6)

where h = 6.6256 × 10−34 J s is Planck’s constant, σ = 1.38054 × 10−23 J/K is the Stefan-Boltzman’s constant, c = 2.998 × 10−8 m/s is the speed of light, θ is the absolute temperature in degrees kelvin, and λ is again the wavelength. Usually objects are not black bodies, and consequently the above law does not apply without certain corrections. Non-black bodies absorb a fraction A, reflect a fraction R, and transmit a fraction T . These fractions are selective, depending on the wavelength and on the angle of incident radiation. By introducing the spectral emissivity ε(λ) to balance the absorbance, it can be found that: A(λ) = ε(λ)

(1.7)

ε(λ) + R(λ) + T (λ) = 1

(1.8)

and

Using these corrections, equation (1.6) can be simplified, yielding: R(λ, θ) = ε(λ)Rblackbody (λ, θ)

(1.9)

This means that the emission coefficient ε(λ) relates the ideal radiation of a black body with real non-black bodies. In summary, an ideal black body is a material that is a


perfect emitter of heat energy, and therefore has the emissivity value equal to unity. In contrast, a material with zero emissivity would be considered a perfect thermal mirror. However, most real bodies, including food objects, show wavelength-dependent emissivities. Since emissivity varies with material, this parameter is the important factor in thermographic image formation. For accurate measurement of temperature, the emissivity should be provided manually to the camera for its inclusion in temperature calculation. The function that describes the thermographic image f (x, y) can be expressed as follows: f (x, y) = f [θ(x, y), ε(x, y)]

(1.10)

where x and y are the coordinates of individual image pixels, θ(x, y) is the temperature of the target at image cooordinates (x, y), and ε(x, y) is the emissivity of the sample also at coordinates (x, y). From the computer vision viewpoint, thermographic images are a function of two variables: the temperature and emissivity variables. Contrast in thermographic images may be the result of either different temperatures of different objects on the scene, or different emissivities of different objects with the same temperature. It can also be the combination of both temperature and emissivity variations. As mentioned previously, the infrared or thermographic cameras operate at wavelengths as long as 14 000 nm (or 14 µm). The infrared sensor array is equivalent to the CCD in the ordinary camera; sensors with a resolution of 160 × 120 pixels or higher are widely available, and their response time is sufficient to provide live thermographic video at 25 frames per second. However, unlike sensors used in conventional imaging systems, the process of image formation and acquisition in thermographic cameras is quite complex. Broadly speaking, thermographic cameras can be divided into two types: those with cooled infrared image detectors and those without cooled detectors. These are discussed in the following section. 3.3.1 Cooled infrared detectors

Cooled IR detectors are typically contained in a vacuum-sealed case and cryogenically cooled. This greatly increases their sensitivity, since their temperature is much lower than that of the objects from which they are meant to detect radiation. Typically, cooling temperatures range from −163◦ C to −265◦ C, with −193◦ C being the most common. In a similar way to common digital cameras, which detect and convert light to electrical charge, the IR detectors detect and convert thermal radiation to electrical signals. In the case of IR cameras, cooling is needed in order to suppress thermally emitted dark currents. A further advantage of cooling is suppression of noise from ambient radiation emitted by the apparatus. Materials used for IR detection include liquidhelium cooled bolometers, photon-counting superconducting tunnel junction arrays, and a wide range of cheaper, narrow-gap semiconductor devices. Mercury cadmium telluride (HgCdTe), indium antimonide (InSb) and indium gallium arsenide (InGaAs) are the most common types of semiconductor IR detectors, with newer compositions such as mercury manganese telluride (HgMnTe) and mercury zinc telluride (HgZnTe) currently being developed. However, the HgCdTe detector and its extension remains the most common IR detector. The principle of operation of an HgCdTe-based detector is illustrated in Figure 1.10.


Vdd Vrst Access Thermal radiation Detector substrate

Access

HgCdTe Indium bumps

C

Silicon multiplexer

Column bus (a)

(b)

Figure 1.10 Hybrid focal plane architecture for HgCdTe-based IR detector showing (a) cell structure and (b) equivalent circuit.

In Figure 1.10, the sensor is represented by a detector diode which is mechanically bonded to a silicon (Si) multiplexer for the read-out operation. An electrical connection is required between each pixel and the rest of the circuitry. This is formed by the heatand pressure-bonding of an indium bump or solder bond. Row and column shift registers allow sequential access to each pixel. Similarly to other semiconductor devices, this type of sensor is constructed using modern fabrication technologies such as vapor deposition epitaxy (Campbell, 2001). In this method, the diode is made by depositing CdTe on sapphire followed by liquid epitaxy growth of HgCdTe. A complete HgCeTe IR detector system usually comprises a small printed circuit board (PCB), complete with a digital signal processor chip (DSP) and an optical system responsible for focusing the scene on to the plane of array. At present, large two-dimensional arrays comprising 2048 × 2048 pixels, with each pixel 18 µm in size, assembled on a 40 × 40-mm device and with a complete infrared camera system, are commercially available. They operate in the bands 3–5 µm or 8–12 µm, and need cooling at −196◦ C. There are different ways to cool the detectors – mainly by using liquefied gas, a cryogenic engine, gas expansion, or the thermoelectric effect. The most common method is cryogenic cooling, employing liquefied gas stored in a vacuum called a Dewar (named after Sir James Dewar, a Scottish scientist who successfully liquefied hydrogen for the first time in 1892). Figure 1.11 shows the construction of a typical Dewar, highlighting all the important elements. Typically, the sensor is mounted directly on the cold surface, with a cold shield and infrared transparent window. Usually a protective coating such as zinc sulfide is applied on to the surface of HgCeTe in order to increase its lifespan. The most commonly used and cheapest liquefied gas is liquid nitrogen, which provides a sustainable cold temperature of −196◦ C without regular filling. Another common method of achieving cooling is through the Joule–Thompson gas expansion method. High-pressure gas such as nitrogen or argon produces droplets of liquid nitrogen at −187◦ C following quick expansion. Compared to the Dewar,


Infrared transparent window Sensor

Cold shield

Liquefied gas

Evacuated space

Figure 1.11 Schematic diagram of a typical Dewar.

this method is noisy and cumbersome. When refilling is not practical, such as for applications in remote areas, a cooling method using a closed Stirling cycle can be employed. This machine cools through the repetitive compression and expansion cycles of a gas piston, and is therefore again cumbersome compared to the Dewar. Another more practical approach to cooling is by thermoelectric elements, based on the Peltier– Thompson effect (Fraden, 1997). This method utilizes a junction of dissimilar metals carrying a current; the temperature rises or falls depending on the direction of the current. Current flowing in one direction results in the Peltier effect, and current flowing in the opposite direction produces the Thompson effect by the same law of physics. Unfortunately, Peltier elements are unattractive for temperatures below −73◦ C owing to high current consumption. In spite of this drawback, the thermoelectric cooling method involves no moving parts, and is quiet and reliable. For these reasons, it is widely used in IR cameras. 3.3.2 Uncooled IR detectors As the name implies, uncooled thermal cameras use sensors that operate at room temperature. Uncooled IR sensors work by changes in resistance, voltage or current when exposed to IR radiation. These changes are then measured and compared with the values at the operating temperature of the sensor. Unlike cooled detectors, uncooled IR cameras can be stabilized at an ambient temperature, and thus do not require bulky, expensive cryogenic coolers. This makes such IR cameras smaller and less costly. Their main disadvantages are lower sensitivity and a longer response time, but these problems have almost been solved with the advent of surface micro-machining technology. Most uncooled detectors are based on pyroelectric materials or microbolometer technology. Pyroelectricity is the ability of some materials to generate an electrical potential when heated or cooled. It was first discovered in minerals such as quartz, tourmaline, and other ionic crystals. The first generation of uncooled thermal cameras looked very similar to the conventional cathode ray tube, apart from the face plate and target material


IR-transparent face plate

IR lens

Signal Pyroelectric electrode plate

x deflection

Mesh

y deflection

Anode

Focusing plate

Electron beam

Modulator Cathode

Video signal Amplifier Figure 1.12

Schematic diagram of the first-generation pyroelectric tube.

(see Figure 1.12). As infrared signals impinge on the pyroelectric plate, the surface temperature of this plate changes. This in turn induces the charge, which accumulates on the pyroelectric material. The electron beam scans this material, and two things may happen depending on whether there is an absence or presence of charge. In the absence of charge (i.e. no radiation), the electron beam is deflected toward the mesh by the action of the x and y deflection plates. In the presence of charge, the electron beam is focused on the spot, thus causing current to flow into an amplifier circuit. In this way a video signal is built as the electron beam scans over the entire surface of the pyroelectric plate. Since the accumulation of charge only occurs when the temperature of the pyroelectric material changes, the pyroelectric tube is only suitable for imaging dynamic occurrences. This effect will benefit certain applications, such as monitoring drying process, where only the fast changes of temperature are recorded (Fito et al., 2004). With the advent of semiconductor technology, it is now possible to produce pyroelectric solid-state arrays with resolution reaching 320 × 240 pixels. This type of camera offers high detectivity, but produces images at a relatively low speed (typically 1 Hz). Furthermore, absolute temperature measurement often requires individual calibration of each element, which significantly slows down the image acquisition time. However, the main advantage lies with its ability to produce an image without the need for cooling. This makes it suitable for a wide range of non-destructive applications, especially in industry. Another type of IR camera is based on microbolometer technology. Theoretically, a microbolometer is a monolithic sensor capable of detecting infrared radiation through the direct or indirect heating of a low-mass, temperature-dependent film. Popular materials include thermistors with high temperature coefficients of resistance, such as vanadium oxide (VOx ), silicon devices such as the Schottky barrier diode and transistor, and thermoelectrics such as the silicon p–n junctions. One example of the bolometer-type uncooled infrared focal plane array (IRFPA), with a 320 × 240-pixel array and operating at a frame rate of 60 Hz, has been investigated for use in industry (Oda et al., 2003).


Beam Bolometer Passivation thin film layers

Diaphragm

Diaphragm

Conducting thin film Reflecting layer

Cavity

ROIC (a)

(b)

Figure 1.13 Schematic representation of a bolometer detector showing (a) the cross-sectional view and (b) the plan view of each bolometer pixel.

Figure 1.13 shows the schematic structure of each bolometer pixel. The pixel is divided into two parts; a silicon readout integrated circuit (ROIC) in the lower part, and a suspended microbridge structure in the upper part. The two parts are separated by a cavity. The microbridge structure is composed of a diaphragm and supported by two beams, thereby thermally isolating the former from the latter heat sink. Manufacture of microbolometers such as the one shown in Figure 1.13 uses microelectromechanical techniques, originally developed at Bell Labs for air-bridge isolation integrated circuits. They are carefully engineered so that part of the IR radiation is absorbed by the silicon passivation layers in the diaphragm and part is transmitted. The transmitted radiation is perfectly reflected by the reflecting layer, and is again absorbed by the passivation layers. In this way, more than 80 percent of the incident IR radiation is absorbed. The absorbed radiation heats the diaphragm and changes the bolometer resistance. Supplying a bias current enables the resistance change to be converted to voltage and detected by ROIC. The analog signal voltage of the ROIC is digitized by analog-to-digital conversion of the receiving circuits. These data are first corrected for non-uniformity in bolometer responsivity, and are then adjusted for video output. The pixel size of such a detector is 37 × 37 µm, and the fill factor is about 72 percent.

3.4 Tomographic imaging While a computer vision system is useful for surface inspections, in many specialized investigations the food technologists and scientists frequently need to “see” an internal view of the sample. It should now be recognized that a clear image of an object’s interior cannot be formed with a conventional imaging instrument because wave motion is continuous in space and time. Wave motion brought to a focus within the region of a particular point necessarily converges before and diverges after it, thereby inherently contaminating the values registered outside that region. Therefore, an image formed of the surface of a body by conventional methods can be clear, but the image depicting the internal structure of the sample will be contaminated. Therefore, the terms “computerassisted tomography” (CAT) and “computed tomography” (CT) emerged following


the development of a CT machine in 1972 at EMI Ltd, by the Nobel Prize winner Geoffrey Hounsfield. This device has revolutionized clinical radiology. Nevertheless, food tomography is a relatively new subject, since such an application requires high expenditure. A typical medical CT scanner can cost tens of millions of pounds, and, with no comparable increase in reimbursement, the purchase of such a system for uses other than medical cannot easily be justified. However, some interesting applications involving the use of tomography for food applications have started to emerge recently, and such tomographic modalities are described here. 3.4.1 Nuclear tomography As the name implies, nuclear tomography involves the use of nuclear energy for imaging the two-dimensional spatial distribution of the physical characteristics of an object, from a series of one-dimensional projections. All nuclear-imaging modalities rely upon acquisition hardware featuring a ring detector which measures the strength of radiation produced by the system. There are two general classes of source of radiation, determined by the degree of control exerted over them by the user. The first class consists of exterior sources (those outside the body), which are usually completely under the control of the experimenter; this method is termed “remote sensing” (see Figure 1.14a). The second group consists of interior sources (those inside the body), which are usually beyond the direct control of the experimenter; this method is termed “remote probing” (see Figure 1.14b). Computed tomography, where radiation is projected into the object, falls into the first category; stimulated emission, as in the case of magnetic resonance imaging (MRI) and in the use of radiopharmaceuticals in single photon-emission

Sample

Sample Detector

Source

(a)

Detector

(b)

(c) Figure 1.14 Two different geometries for tomographic imaging: (a) remote sensing and (b) remote probing. (c) Typical scanning pattern showing two orthogonal projections.


computed tomography (SPECT) and positron-emission tomography (PET), fall into the second category. Regardless of the scanning geometry, tomographic imaging shares one common feature: the requirement to perform complex mathematical analysis of the resulting signals using a computer. There are many good reviews on this subject, and interested readers are referred to publications by Brooks and Di Chiro (1975, 1976), and Kak (1979). Here, a brief description of the various tomographic modalities is provided, focusing on the advancement of the technology since its inception more than 30 years ago.

3.4.1.1 Computed tomography (CT) As shown in Figure 1.14, essentially CT involves scanning the source and detector sideways to produce single-projection data. This procedure is repeated at many viewing angles until the required set of all projection data is obtained. Image reconstruction from the data remains one of the important tasks in CT that can be performed using a variety of methods. The history of these reconstruction techniques began in 1917 with the publication of a paper by the Austrian Mathematician J. Radon, in which he proved that a two-dimensional or three-dimensional object can be reconstructed uniquely from the infinite set of all its projections (Herman, 1980). To date, there have been hundreds of publications on computed tomography imaging. A good summary is provided by Kak and Slaney (1988). When the first CT machines were introduced in 1972, the spatial resolution achievable was three line pairs per millimeter, on a grid of 80 × 80 per projection. The time taken to perform each projection scan was approximately 5 minutes. In contrast, a modern machine achieves 15 line pairs per millimeter, on a grid of 1024 × 1024 per projection, with a scan time per projection of less than 1 second. The projection thickness typically ranges from 1 to 10 mm, and the density discrimination achievable is better than 1 percent. These machines use an X-ray source which rotates in a circular path around the sample. A collimator is employed in order to produce a sharp, pencil-beam X-ray, which is measured using detectors comprising a static ring of several hundreds of scintillators. These have sometimes been constructed from xenon ionization chambers, but a more compact solution is offered by solid-state systems, where a scintillation crystal is closely coupled to a photodiode. This source–detector combination measures parallel projections, one sample at a time, by stepping linearly across the object. After each projection, the gantry rotates to a new position and these procedures are repeated until data are gathered at sufficient viewing angles. The latest generation of CT machines employs a fan-beam arrangement as opposed to parallel-beam geometry. In this way, the size of the beam can be enlarged to cover the object field of view. Consequently, the gantry needs only to rotate, thus speeding-up the acquisition time. Employing a stationary ring comprising, typically, 1000 detectors, the data acquisition time of a modern CT scanner is generally less than 0.1 s. Figure 1.15 illustrates the essential elements of such systems. Since CT is based upon the attenuation of X-rays, its primary strength is the imaging of calcified objects such as bone and the denser tissues. This limits its applications in food technology, since food objects are mostly soft or semi-fluid. This, as well as the


Motion controller

Data-acquisition system Fan beam

Imagereconstruction system Computer for display and control

Ring detectors Figure 1.15

Modern CT usually employs fan-beam geometry in order to reduce the data-capturing speed.

expense, is the reason that CT imaging was initially limited to medical applications. However, in the 30 years since its inception, its capabilities and applications have been expanded as a result of the advancement of technology and software development. While medical disorders are still a common reason for CT imaging, many other scientific fields – such as geology, forestry, archaeology, and food science – have found CT imaging to be the definitive tool for diagnostic information. For instance, CT combined with appropriate image analysis has been used to study the magnitude and gradients of salt in dry-cured ham in the meat water phase (Vestergaard et al., 2005). In studying growth and development in animals, Kolstad (2001) used CT as a non-invasive technique for detailed mapping of the quantity and distribution of fat in crossbred Norwegian pigs. There are other recent applications involving CT in agriculture and food tomography, and interested readers are again directed to relevant publications (see, for example, Sarigul et al., 2003; Fu et al., 2005; Babin et al., 2006). 3.4.1.2 Magnetic resonance imaging (MRI) Previously known as nuclear magnetic resonance (NMR) imaging, MRI gives the density of protons or hydrogen nuclei of the body at resonant frequency. Unlike CT, MRI provides excellent renditions of soft and delicate materials. This unique characteristic makes MRI suitable for visualization of most food objects, and applications range from non-invasive to real-time monitoring of dynamic changes as foods are processed, stored, packaged, and distributed. Hills (1995) gives an excellent review on MRI applications from the food perspective. In principle, MRI is based on the association of each spatial region in a sample with a characteristic nuclear magnetic resonance frequency, by imposing an external magnetic field. Without the external magnetic field, the magnetic moment would point in all directions at random, and there would be no net magnetization. However, in the presence of a large magnetic field, the hydrogen nuclei will preferentially align their spin in the direction of the magnetic field. This is known as the Lamor effect, and the frequency at which the nucleus proceeds around the axis is termed the Lamor


Superconducting coil

RF source

RF excitation coil

RF receiver

RF reception coil

Signal conditioning

Computer for control and image reconstruction

Figure 1.16 Block diagram of a typical MRI system.

frequency (McCarthy, 1994). This effect implies a transfer of energy from the spin system to another system or lattice. The transfer of energy is characterized by an exponential relaxation law with time constants T1 and T2 , which are also known as the spin–lattice excitation and spin–spin relaxation times, respectively (McCarthy, 1994). In commercial MRI, the magnetic field ranges from 0.5 to 2.0 tesla (compared with Earth’s magnetic field of less than 60 µT). T1 is typically of the order of 0.2–2 s, and T2 ranges from 10 to 100 ms. According to Planck’s equation E = hf, for a field strength of 1.5 T, f corresponds to radiowaves with a frequency of 60 MHz. This is the resonant frequency of the system. Therefore, by applying a radio-frequency (RF) field at the resonant frequency, the magnetic moments of the spinning nuclei lose equilibrium and hence radiate a signal which is a function of the line integral of the magnetic resonance signature in the object. This radiation reflects the distribution of frequencies, and a Fourier transform of these signals provides an image of the spatial distribution of the magnetization (Rinck, 2001). A basic block diagram of a typical MRI data-acquisition system is shown in Figure 1.16. In general, the MRI system comprises a scanner, which has bore diameter of a few tens of centimeters; a static magnetic field, which is generated by a superconducting coil; and RF coils, which are used to transmit radio-frequency excitation into the material to be imaged. This excites a component of magnetization in the transverse plane which can be detected by a RF reception coil. The signals are transduced and conditioned prior to image reconstruction. Current MRI scanners generate images with sub-millimeter resolution of virtual slices through the sample. The thickness of the slices is also of the order of a millimeter. Contrast resolution between materials depends strongly on the strength of the magnetization, T1 , T2 , and movement of the nuclei during imaging sequences. The most striking artefacts appear when the magnetic field is disturbed by ferromagnetic objects. Other artefacts, such as ringing, are due to the image reconstruction algorithm and sensor dynamics. Owing to the fact that MRI provides rapid, direct, and, most importantly, noninvasive, non-destructive means for the determination of not only the quantity of


water present but also the structure dynamic characteristics of the water, this relatively new imaging technique has become useful for food engineering. There are numerous applications of MRI, since water is the basic building block of many food materials. Figure 1.17 shows examples of MRI-captured images within corn kernels during the freezing process (Borompichaichartkul et al., 2005). The brighter areas show locations where proton mobility is high, and thus water exists as a liquid. In this example, MRI provides useful information for characterizing the physical state of water in frozen corn. Other interesting applications include real-time monitoring of ice gradients in a doughstick during the freezing and thawing processes (Lucas et al., 2005), mapping the temperature distribution patterns in food sauce during microwave-induced heating (Nott and Hall, 1999), and predicting sensory attributes related to the texture of cooked potatoes (Thybo et al., 2004). These examples – a far from exhaustive list – serve to emphasize the potential of MRI for revolutionizing food science and engineering. As with CT imagers, the major drawback is the current expense of an MRI machine – typically between £500 000 and £1 million. Consequently, at present MRI machines are only used as a research and development tool in food science. In order for MRI to be applied successfully on commercial basis, the possible benefits must justify the expense. However, the rapidly decreasing cost of electronic components, combined with the ever-increasing need for innovation in the food industry, indicate that it should not be too long before a commercial and affordable MRI machine is developed for food quality control.

25οC

⫺10οC

⫺20οC

18.6%

30.8%

(a)

(b)

(c)

Figure 1.17 Examples of MRI images showing the distribution of water and its freezing behavior in different areas within the corn kernels: (a) images captured before freezing at different moisture contents; (b) and (c) images acquired at specified temperatures and moisture content levels (Borompichaichartkul et al., 2005).


3.4.2 Electrical tomography

Unlike nuclear imaging, electrical tomography (ET) uses electrical signals in the form of voltage and current of a magnitude of less then tens of millivolts and milliamperes, respectively. Therefore, the method is inherently safe and requires no expensive and complicated hardware. Sensing modalities include electrical-resistant tomography (ERT), electrical-capacitance tomography (ECT), and microwave tomography (MT). There are a few other modalities, but the ERT and MT techniques have been successfully applied to food imaging, and therefore this discussion will focus on these two imaging modalities only. There is much literature on this subject, but imaging examples provided here are based on work by Henningsson et al. (2005), who investigated the use of the ERT technique for yoghurt profiling, and on recent research in applying MT for grain imaging (Lim et al., 2003). Both ERT and MT are soft-field sensor systems, since the sensing field is altered by the density distribution and physical properties of the object being imaged. Therefore, as previously discussed, this limits the resolution compared to hard-field sensors. Nevertheless, these relatively new imaging modalities are useful for some specialized food applications where low-imaging resolution is adequately acceptable. As shown in Figure 1.18, ERT and MT tomographic systems generally can be subdivided into three basic parts: the sensor, the data-acquisition system, and the image-reconstruction interpretation and display. In order to perform imaging, an electrical signal is injected into a reactor through an array of sensors which are mounted non-invasively on the reactor surface, where the response of the system is measured. In the case of ERT, a low-frequency AC current is injected and voltages are measured; in MT, the reactor is irradiated with microwave signals and the transmitted or scattered fields (or both) are measured. There are many ways that sensors can be configured to do the measurement. The ERT system, employing a

Cross-section through reactor Sensor sites

Reactor wall

Data-acquisition system

Sensors on reactor Electrical signals Food reactor

Figure 1.18 Schematic block diagram of a typical ET instrument.

Imagereconstruction system


four-electrode measurement protocol, uses one pair of adjacent sensors to inject current, and voltages appearing at all the other pairs of adjacent sensors are measured. The number of independent measurements obtained using this geometry with N sensors is determined to be equal to N (N2− 3) (Barber and Brown, 1984). Similarly, the number of unique measurements in MT also depends on the sensor geometry. In the case of multiple-offset geometry with N transmitters and M receivers, the total number of measurement is MN (Lim et al., 2003). Using suitable reconstruction methods, the measured data can be processed, delivering a two-dimensional image depicting the conductivity or permittivity distributions in ERT or MT, respectively. By using the information from several sensor planes, a three-dimensional reconstruction can be interpolated across the sectional map (Holden et al., 1998). The general applicability of ERT is illustrated by the work of Henningsson et al. (2005), who studied velocity profiles of yoghurt and its rheological behavior in a pipe of industrial dimensions. A cross-correlation technique was used to transform the dualplane conductivity maps into velocity profiles. Comparing simulated and experimental results, they discovered that ERT results have some noise (and thus uncertainty) in the region near the wall, but the centerline velocities are very well resolved with an error of less than 7 percent. They concluded that ERT is a useful method for determination of the velocity profile of food; the information produced can be used in process conditioning in order to minimize loss of product. Meanwhile, Lim et al. (2003) exploited the sensitivity of microwave signals to permittivity perturbation, which allowed them to apply MT measurements for mapping the moisture profiles in grain. Image reconstruction was based on the optical approach, permitting the use of straight-ray approximation for data inversion. Examples of moisture tomograms obtained using this method are illustrated in Figure 1.19. Tests indicate that this imaging modality would considerably

Moisture (%) 18.4

Moisture (%) 12.4 20.1

(a)

Moisture (%) 12.4 24.8

(b)

12.4

(c)

Figure 1.19 Example of MT images reconstructed from grain with homogeneous moisture of 12.4%. Higher moisture anomalies were simulated at the left-centre of the cross-section, having values of (a) 18.3%, (b) 20.1%, and (c) 24.8%.

Nomenclature 31

enhance results in situations where large dielectric constant differences exist in moisture regimes, such as mixtures of water and ice. For certain moisture regimes where the difference in dielectric constant still exists but is small, it is important to consider the electric field distortion due to diffraction and scattering effects, and to account for these in the reconstruction.

4 Conclusions As discussed above, there are several powerful imaging modalities that are capable of producing food images, each having particular strengths and weaknesses. CCD vision systems, covering both the visible and infrared regions, are suitable for surface imaging, while CT, MRI, and ET are oriented for imaging internal structures. Of the latter three, CT is suitable for imaging hard and solid objects, MRI for functional imaging, and ET for conductivity or permittivity mapping. Some of these technologies are already available commercially, while some are still in the development stage. Currently under development is a system that can combine results from various modalities in order to enhance and improve image quality further. With careful calibration, images from different modalities can be registered and superimposed, giving rise to what is presently known as “multimodal imaging” or the “sensor fusion technique.” With intense research being pursued in some of the world’s leading laboratories, it will not be long before such an emerging technology reaches food technologists and scientists.

Nomenclature θ λ v ρ σ c E f h T1 T2 Z

absolute temperature, K wavelength, m speed of sound, m/s density, kg/m3 Stefan-Boltzman constant, 1.38054 × 10−23 J/K speed of light, 2.998 × 108 m/s Energy, J frequency, Hz Planck’s constant, 6.626076 × 10−34 J s excitation time, s relaxation time, s acoustical impedance,

Abbreviations: AC alternating current A/D analog-to-digital converter CAT computer-assisted tomography CCD charge couple device


CCIR Comite Consultatif International des Radiocommunication CdTe Cadmium telluride CMOS Complementary metal oxide silicon CT computed tomography DC direct current DSP digital signal processor ECT electrical-capacitance tomography ERT electrical-resistant tomography EUV extreme ultraviolet FET Field effect transistor fps frames per second FUV far ultraviolet GPR ground probing radar HgCdTe Mercury cadmium telluride HgMnTe Mercury manganese telluride InGaAs Indium gallium arsenide InSb Indium antimonide IR infrared IRFPA infrared focal plane array LED light-emitting diode MRI magnetic resonance imaging MT microwave tomography NIR near infrared NMR nuclear magnetic resonance NUV near ultraviolet PC personal computer PCI peripheral component interface PET positron-emission tomography RF radio frequency ROIC read-out integrated circuit Rx receiver Si silicon SPECT single photon-emission computed tomography TOF time of flight Tx transmitter UV ultraviolet VOx vanadium oxide

References Abdullah MZ, Abdul-Aziz S, Dos-Mohamed AM (2000) Quality inspection of bakery products using color-based machine vision system. Journal of Food Quality, 23, 39–50.

References 33

Abdullah MZ, Fathinul-Syahir AS, Mohd-Azemi BMN (2005) Automated inspection system for color and shape grading of starfruit (Averrhoa carambola L.) using machine vision sensor. Transactions of the Institute of Measurement and Control, 27 (2), 65–87. Abdullah MZ, Guan LC, Mohd-Azemi, BMN (2001) Stepwise discriminant analysis for color grading of oil palm using machine vision system. Transactions of IChemE, Part C, 57, 223–231. Babin P, Della Valle G, Chiron H, Cloetens P, Hoszowska J, Penot P, Réguerre AL, Salva L, Dendieval R (2006) Fast X-ray tomography analysis of bubble growth and foam setting during bread making. Journal of Cereal Science, 43 (3), 393–397. Barber DC, Brown BH (1984) Applied potential tomography. Journal of Physics E Scientific Instrument, 11 (Suppl A), 723–733. Batchelor BG (1985) Lighting and viewing techniques. In Automated Visual Inspection (Batchelor BG, Hill DA, Hodgson DC, eds). Bedford: IFS Publication Ltd, pp. 103–179. Borompichaichartkul C, Moran G, Srzednicki G, Price WS (2005) Nuclear magnetic resonance (NMR) and magnetic resonance imaging (MRI) studies of corn at subzero temperatures. Journal of Food Engineering, 69 (2), 199–205. Brooks RA, Di Chiro G (1975) Theory of image reconstruction in computed tomography. Radiology, 117, 561–572. Brooks RA, Di Chiro G (1976) Principles of computer assisted tomography (CAT) in radiographic and radioisotope imaging. Physics in Medical Biology, 21 (5), 689–732. Camarena F, Martínez-Mora JA (2005) Potential of ultrasound to evaluate turgidity and hydration of orange peel. Journal of Food Engineering, 75 (4), 503–507. Campbell SA (2001) The Science and Engineering of Microelectronic Fabrication. New York: Oxford University Press. Daley W, Carey R, Thompson C (1993) Poultry grading inspection using colour imaging. SPIE Proceedings Machine Vision Applications in Industrial Inspection, 1907, 124. Doebelin EO (1996) Measurement Systems: Application and Design. New York: McGraw Hill. Fishbane PM, Gasiorowiczs S, Thornton ST (1996) Physics for Scientists and Engineers. Upper Saddle River: Prentice-Hall. Fito PJ, Ortolá MD, de los Reyes R, Fito P, de los Reyes E (2004) Control of citrus surface drying by image analysis of infrared thermography. Journal of Food Engineering, 61 (3), 287–290. Fraden J (1997) Handbook of Modern Sensors: Physics, Designs and Applications. New York: American Institute of Physics Press. Fu X, Milroy GE, Dutt M, Bentham AC, Hancock BC, Elliot JA (2005) Quantitative analysis of packed and compacted granular systems by x-ray microtomography. SPIE Proceedings Medical Imaging and Image Processing, 5747, 1955. Gan TH, Pallav P, Hutchins DA (2005) Non-contact ultrasonic quality measurements of food products. Journal of Food Engineering, 77 (2), 239–247. Gaossorgues G (1994) Infrared Thermography. London: Chapman and Hall. Ginesu G, Guisto DG, Märgner V, Meinlschmidt P (2004) Detection of foreign bodies in food by thermal image processing. IEEE Transactions of Industrial Electronics, 51 (2), 480–490.


Gómez AH, He Y, Pereira AG (2005) Non-destructive measurement of acidity, soluble solids and firmness of Satsuma mandarin using Vis/NIR-spectroscopy techniques. Journal of Food Engineering, 77 (2), 313–319. Gunasekaran S (1996) Computer vision technology for food quality assurance. Trends in Food Science & Technology, 7, 245–256. Heinemann PH, Hughes R, Morrow CT, Sommer III HJ, Beelman RB, Wuest PJ (1994) Grading of mushrooms using machine vision system. Transactions of the ASAE, 37 (5), 1671–1677. Henningsson M, Ostergren K, Dejmek P (2005) Plug flow of yoghurt in piping as determined by cross-correlated dual-plane electrical resistance tomography. Journal of Food Engineering, 76 (2), 163–168. Herman G (1980) Image Reconstruction from Projections. New York: Academic Press. Hills B (1995) Food processing: an MRI perspective. Trends in Food Science & Technology, 6, 111–117. Holden PJ, Wang M, Mann R, Dickin FJ, Edwards RB (1998) Imaging stirred-vessel macromixing using electrical resistant tomography. AIChE Journal, 44 (4), 780–790. Kak, CK (1979) Computerized tomography with x-ray, emission and ultrasound sources. Proceedings of IEEE, 67 (9), 1245–1272. Kak AC, Slaney M (1988) Principles of Computerized Tomography Imaging. New York: IEEE Press. Kolstad K (2001) Fat deposition and distribution measured by computer tomography in three genetic groups of pigs. Livestock Production Science, 67, 281–292. Lim MC, Lim KC, Abdullah MZ (2003) Rice moisture imaging using electromagnetic measurement technique. Transactions of IChemE, Part C, 81, 159–169. Lucas T, Greiner A, Quellec S, Le Bail A, Davanel A (2005) MRI quantification of ice gradients in dough during freezing or thawing processes. Journal of Food Engineering, 71 (1), 98–108. Matas J, Marik R, Kittler J (2005) Color-based object recognition under spectrally nonuniform illumination. Image and Vision Computing, 13 (9), 663–669. McCarthy M (1994) Magnetic Resonance Imaging in Foods. NewYork: Chapman and Hall. McClements DJ (1995) Advances in the application of ultrasound in food analysis and processing. Trends in Food Science and Technology, 6, 293–299. Morlein D, Rosner F, Brand S, Jenderka KV, Wicke M (2005) Non-destructive estimation of the intramuscular fat content of the longissimus muscle of pigs by means of spectral analysis of ultrasound echo signals. Meat Science, 69, 187–199. Nott KP, Hall LD (1999) Advances in temperature validation of foods. Trends in Food Science & Technology, 10, 366–374. Oda N, Tanaka Y, Sasaki T, Ajisawa A, Kawahara A, Kurashina S (2003) Performance of 320 × 240 bolometer-type uncooled infrared detector. NEC Research and Development, 44 (2), 170–174. Paulsen M (1990) Using machine vision to inspect oilseeds. INFORM, 1 (1), 50–55. Pearson T (1996) Machine vision system for automated detection of stained pistachio nuts. Lebensmittel Wissenschaft und Technologie, 29 (3), 203–209. Pedreschi F, León J, Mery D, Moyano P (2006) Development of a computer vision system to measure the color of potato chips. Food Research International, 39, 1092–1098.

References 35

Rinck PA (2001) Magnetic Resonance in Medicine. Berlin: Blackwell. Sarigul E, Abott AL, Schmoldt DT (2003) Rule driven defect detection in CT images of hardwood logs. Computers and Electronics in Agriculture, 41, 101–119. Simal S, Benedito J, Clemente G, Femenia A, Roselló C (2003) Ultrasonic determination of the composition of a meat-based product. Journal of Food Engineering, 58 (3), 253–257. Steele DJ (1974) Ultrasonics to measure the moisture content of food products. British Journal of Non-destructive Testing, 16, 169–173. Steinmetz V, Crochon M, Bellon-Maurel V, Garcia-Fernandez JL, Barreiro-Elorza P, Vestreken L (1996) Sensors for fruit firmness assessment: comparison and fusion. Journal of Agricultural Engineering Research, 64 (1), 15–28. Stiles WS, Wyszecki G (2000) Color Science: Concepts and Methods: Quantitative Data and Formulas. New York: Wiley Interscience Publishers. Tao Y, Heinemann PH, Varghese Z, Morrow CT, Sommer III HJ (1995) Machine vision for color inspection of potatoes and apples. Transactions of the ASAE, 38 (5), 1555–1561. Thybo AK, Szczpinski PM, Karlsoon AH, Donstrup S, Stodkilde-Jorgesen HS, Andersen HJ (2004) Prediction of sensory texture quality attributes of cooked potatoes by NMR-imaging (MRI) of raw potatoes in combination with different image analysis method. Journal of Food Engineering, 61 (1), 91–100. Vestergaard C, Erbou SG, Thauland T, Adler-Nisen J, Berg P (2005) Salt distribution in dry-cured ham measured by computed tomography and image analysis. Meat Science, 69, 9–15. Wells PNT (1969) Physical Principles of Ultrasonic Diagnosis. NewYork: Academic Press.

Image Segmentation Techniques Chaoxin Zheng and Da-Wen Sun Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland

1 Introduction Owing to the imperfections of image acquisition systems, the images acquired are subject to various defects that will affect the subsequent processing. Although these defects can sometimes be corrected by adjusting the acquisition hardware, for example by increasing the number of images captured for the same scene and adopting higher quality instruments, such hardware-based solutions are time-consuming and costly. Therefore it is preferable to correct the images, after they have been acquired and digitized, by using computer programs, which are fast and relatively low-cost. For example, to remove noise, smooth filters (including linear and median filters) can be applied; to enhance contrast in low-contrast images, the image histograms can be scaled or equalized. Such corrections of defects in images are generally called “image pre-processing.” After pre-processing, the images are segmented. Segmentation of food images, which refers to the automatic recognition of food products in images, is of course required after image acquisition, because food quality evaluation is completely and automatically conducted by computer programs, without any human participation in computer vision techniques. Although image segmentation is ill-defined, it can generally be described as separating images into various regions in which the pixels have similar image characteristics. Since segmentation is an important task, in that the entire subsequent interpretation tasks (i.e. object measurement and object classification) rely strongly on the segmentation results, tremendous efforts are being made to develop an optimal segmentation technique, although such a technique is not yet available. Nevertheless, a large number of segmentation techniques have been developed. Of these, thresholding-based, region-based, gradient-based, and classification-based segmentation are the four most popular techniques in the food industry, yet none of these can perform with both high accuracy and efficiency across the wide range of different food products. Consequently, other techniques combining several of the above are also Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


2

38 Image Segmentation Techniques

being developed, with a compromise on accuracy and efficiency. Even so, they are not adaptable enough for use on the full diversity of food products. This chapter reviews the image pre-processing techniques and the image segmentation techniques that are adoptable or have already been adopted in the food industry. The feasibility of the various techniques is also discussed. This review can serve as a foundation for applying the segmentation techniques available, and for the development of new segmentation techniques in computer vision systems.

2 Pre-processing techniques 2.1 Noise removal Images captured using various means are all subject to different types of noise, such as the read-out noise while reading information from cameras, the wiring noise while transferring video signals from cameras to computers, and the electronic noise while digitizing video signals. All these lead to degradation of the quality of the images when they are subsequently processed. In Figure 2.1, two images of the same scene have been taken at an interval of less than 2 seconds, using the same image acquisition system, and the differences are illustrated to demonstrate the noise produced during image acquisition. It is clearly important that noise is removed after images have been digitized and stored in computers, and the most efficient and feasible approach for image noise removal is to “average” the image by itself. 2.1.1 Linear filter The simplest method of averaging an image by itself is the linear filter, by which the intensity values of pixels in the image are averaged using the intensity values of their neighboring pixels within a small region. The filter processing can be described by the following equation: +M +M +M +M f (x, y) = w i, j f (x + i, y + j) w i, j (2.1) i=−M j=−M

i=−M j=−M

where f (x, y) is the intensity value of pixel (x, y), while M is the size of the filter and w represents the weighting of the filter. The weighting and size of the filter can be adjusted to remove different types of noise. For instance, increasing the weighting of the central pixel means that the central pixel dominates the averaging. Increasing the size of the filter results in a smoother image with less noise, but the detail of the image is reduced. 2.1.2 Median filter Another popular filter that is widely used is the median filter. The intensity values of pixels in a small region within the size of the filter are examined, and the median intensity value is selected for the central pixel. Removing noise using the median filter does not reduce the difference in brightness of images, since the intensity values of the filtered image are taken from the original image. Furthermore, the median filter does not shift the edges of images, as may occur with a linear filter (Russ, 1999). These

Pre-processing techniques 39

(a)

(b)

(c)

(d) Figure 2.1 Illustration of noise present in images: (a) two-color peanut images (in RGB space) taken at an interval of less than 2 seconds; (b) their difference in the red component; (c) their difference in the green component; (d) their difference in the blue component. Contrast was enhanced in images (b), (c), and (d).

two primary advantages have led to great use of the median filter in the food industry (Du and Sun, 2004, 2006a; Faucitano et al., 2005).

2.2 Contrast enhancing Sometimes images captured are of low contrast – in other words, the intensity values of the images are within a small range of intensity levels, and thus pixels with different


(a) Figure 2.2

(b)

Illustrations of (a) low-contrast image, and (b) high contrast after histogram scaling.

intensity values are not well distinguished from each other. An image in which the intensity values range from 100 to 109 is shown in Figure 2.2a. However, it is impossible to sense the difference of intensity values between pixels. The process of contrastenhancing is designed to increase the difference in intensity values among pixels so that they can be effortlessly distinguished by human or computer vision. Most of the contrast-enhancing utilizes the image histogram, which is a plot showing the occurrence of intensity values in images (Jain, 1989). 2.2.1 Histogram scaling

In histogram scaling, the original histogram is transferred from one scale to another – mostly from a smaller scale to larger one. Accordingly, the difference between two neighboring intensity values is increased. For instance, Figure 2.2b is the transformed image of Figure 2.2a, whose histogram has been reallocated from [100, 109] to the scale of [0, 200] linearly so that the difference between neighboring intensity values of the original image is increased from 1 to 20 – which can easily be observed. The transform function used for histogram scaling can be linear or non-linear, and one-to-one or multiple-to-one. 2.2.2 Histogram equalization

Most of the transform functions for histogram scaling are limited to proposed cases. Therefore, it is important to develop a flexible and hopefully optimal function that can be employed for different types of images. Taking this into consideration, histogram equalization has been developed, in which a much more uniform histogram is generated from the original histogram by spreading out the number of pixels at the histogram peaks and selectively compressing those at the histogram valleys (Gauch, 1992). Histogram equalization can be simply described by equation (2.2):

j =

j i=l

H(i)

L i=l

H(i)

(2.2)

Segmentation techniques 41

where H denotes the original histogram, and l and L are the minimum and maximum intensity values, respectively. The parameter i is the ith intensity value in the histogram; j and j stand for the intensity value in the original histogram, and its corresponding intensity value in the equalized histogram, respectively. Sometimes the contrast needs to be constrained to a limited range for the purpose of retaining visual information of objects in images, especially those with homogeneous intensity values. Therefore, the contrast-limited adaptive histogram equalization method was developed and has been applied to adjust pork images by facilitating the segmentation of pores (Du and Sun, 2006a). In this method, the contrast of the images is enhanced by first dividing each image into non-overlapping small regions, and then enhancing the contrast in each small region.

3 Segmentation techniques 3.1 Thresholding-based segmentation In thresholding-based segmentation the image histogram is partitioned into two classes using a single value, called bi-level thresholding (Figure 2.3), or into multiple classes using multiple values, called multilevel thresholding, based on the characteristics of the histogram. In bi-level thresholding, pixels with intensity values less than the threshold are set as background (object) while others are set as object (background). In multiplelevel thresholding, pixels with intensity values between two successive thresholds are assigned as a class. However, in tri-level thresholding, only two classes are normally defined – i.e. one with intensity values between the two thresholds, and the other with intensity values outside the two thresholds. Theoretically, the levels of thresholding can be increased limitlessly according to the number of objects present in images; however, the computation load will be increased exponentially. For example, for searching the four-level thresholding in a gray image, the calculation would be as large as O (L3 ), where L is the gray level of the image (typically 256 for a gray image). The large calculation means that multilevel (more than tri-) thresholding is unfeasible, and therefore only bi-level and tri-level thresholding are used in practice. It is obvious that the threshold for the segmentation described above is a fixed value (called the global threshold) across the whole image. There is another kind of threshold, called the local threshold, which is an adaptive value determined by the local characteristics of pixels. However, only the global threshold is popularly used in the food industry, mainly because the global threshold is selected from the image histogram rather than the image itself. Therefore the computing speed is not affected by the image size, as might be the case in local-threshold methods. As the adaptive threshold is hardly used in the food industry, it is not further discussed here. However, for the segmentation of complex food images, such as toppings of pizzas (see Figure 2.4; Sun, 2000), the global threshold is not competent. One explanation for this is that the number of classes defined by the global threshold is restricted to two (object and background), which is far less than those required to segment the complex food images, since there are many food products with different intensity-level values to be segmented.


(a)

(c)

Occurrence of intensities

Background

Object

Threshold

0

Intensities of image (b)

Figure 2.3 Thresholding the histogram of a beef image: (a) image of beef; (b) thresholding the histogram; (c) binarized (a) by the threshold.

3.1.1 Threshold selection There are four main methods or algorithms for the selection of the global threshold: manual selection, isodata algorithm, objective function, and histogram clustering.

3.1.1.1 Manual selection The simplest global thresholding method is by manual selection, in which the threshold is manually selected by researchers using graphic–user interface image-processing software such as Photoshop (Adobe Systems Incorporated, USA), Aphelion (AAI, Inc., USA), Optimas (Media Cybernetics, Inc., USA), etc. Although this method is the simplest and easiest in implementation, it is not ideal for online automatic food-quality evaluation using computer vision without any human participation. Therefore, methods for automatically selecting a threshold have been developed.


(a)

(b)

Figure 2.4 Images of pizza toppings: (a) original image; (b) segmented image of (a).

3.1.1.2 Isodata algorithm The first automatic threshold selecting method was probably by isodata algorithm, which was originally proposed by Ridler and Calvard (1978). In the algorithm, a threshold is first guessed (in most cases it is selected by the average intensity value of the image) and then used to segment the histogram into two classes, i.e. A and B. The average intensity values, mA and mB , for both classes are calculated, and the new threshold is then determined as the average of mA and mB . The new threshold is updated iteratively by the new average intensity values until convergence is achieved. Alternatively, the objective function method might be used. Here, the histogram is preliminarily normalized and regarded as probability distributions using equation (2.3): L h( j) = H( j) H(i) (2.3) i=l

The distribution is classified into two groups (i.e. objects and background) using a threshold, which is an intensity value iteratively selected from the minimum to the maximum of the intensity values. The optimal threshold is determined as the one that maximizes the objective function, and is based on the interaction of the two classes with regard to evaluating the success of the thresholds. Two kinds of objective functions are mostly used: variance-based and entropy-based. In the variance-based objective function (Otsu, 1979), the optimal threshold t is selected to maximize the between-class variance, which can be calculated by σ=

[µ(L)ω(t) − µ(t)]2 ω(t)[1 − ω(t)]

(2.4)

where ω and µ are the zero-th- and first-order cumulatives of the probability distribution, respectively. In the entropy-based objective function, the optimal threshold is selected as the intensity value at which the sum entropies of the two classes are maximized. However,


the different calculation of the sum entropy leads to different entropy thresholding methods, as in those proposed by Pun (1980), Kapur et al. (1985), Sahoo et al. (1997), etc. Researchers have undertaken the comparison of these two objective functions. However, most of the comparisons are based on practice – in other words, the performance of these two objective functions is compared by applying them respectively to segment a set of images. No theoretical comparison has so far been conducted, and thus the comparison results are dependent on the set of images being used. Nevertheless, some advantages and disadvantages of the two methods have already been found. It is suggested that the variance-based objective function generally performs better than the entropy-based one, except for images in which the population (the number of pixels of one class) of one class is relatively larger than that of the other (Read, 1982). The worst situation, that the variance-based objective function will produce erroneous results, occurs in images in which the ratio of the population of one class over the other is lower than 0.01 (Kittler and Illingworth, 1985). In contrast, the entropy-based objective functions retain a more stable performance across images with different ratios of population, yet there is a major problem with entropy-based methods. When the probability distribution of an intensity value is too small, the entropy of the value is exponentially larger than those of other values, which will introduce potentially large computation errors (Sahoo et al., 1997). Therefore, the threshold selected will be much less reliable. 3.1.1.3 Histogram clustering The clustering method that is mainly used in threshold selection is k-means clustering. An intensity value from l to L is picked as the threshold to segment the histogram into two classes, object and background, with mean intensity values of mA and mB . If the threshold satisfies the criterion that every intensity value of class A (B) is closer to mA (mB ) than to mB (mA ), the threshold is selected as a candidate threshold. Afterwards, partition error of each candidate threshold is computed using equation (2.5), and the one with the smallest partition error is chosen as the optimal threshold. e=

L

H(i)[i − µ(t)]

(2.5)

i=l

3.1.1.4 Other techniques Besides the techniques described above, there are many other thresholding-based segmentation techniques – for example, the minimum error technique (Kittler and Illingworth, 1986), the moment-preserving technique (Tsai, 1985), the window extension method (Hwang et al., 1997), and the fuzzy thresholding technique (Tobias and Seara, 2002). As these techniques are less popular and much more complex than the isodata algorithm, objective function, and histogram clustering methods, they are only mentioned here for completeness. Among the above automatic threshold selection methods, there is no single one that can perform better overall than any of the others. Therefore, it is recommended that several methods be proposed to identify the one with the best performance. Furthermore, for the purpose of eliminating the effects of noise in segmentation, twodimensional histogram thresholding can be proposed. The two-dimensional histogram


L

C

B

A

D

k

l

L

Figure 2.5 Illustration of thresholding on a two-dimensional histogram (Zheng et al., 2006). Region A is regarded as being object (background), and B as being background (object). Regions C and D are referred to as noises and edges, and thus are ignored in threshold selection.

is constructed by considering the co-occurrence of the intensity values of pixels, and the average intensity values between pixels and their neighboring pixels (Abutaleb, 1989). The threshold for a two-dimensional histogram is illustrated in Figure 2.5. Although two-dimensional thresholding performs better than one-dimensional thresholding, a far greater computation load is required for the two-dimensional technique; for this reason, it is less popular in the food industry. Although the techniques described above are all bi-level thresholding, apart from the isodata algorithm, most of them can be easily expanded to tri-level thresholding simply by increasing the number of classes segmented by the threshold to three – object, background1, and background2 (or object1, object2, and background). 3.1.2 Image-opening and -closing After image thresholding, some defects might be present in the images – for example, some parts of objects might be misclassified as background, and some small regions of background might be mistakenly segmented as objects. Consequently, image-opening and image-closing are proposed for post-processing images segmented by thresholding. Image opening involves reserving the unsegmented parts of objects using first image dilation, by merging neighboring pixels of an object into the object, and then image erosion, by removing boundary pixels from the object. On the contrary, image-closing is image erosion followed by image dilation in order to eliminate the unsegmented parts of the background. An example is provided in Figure 2.6. To remove small defects, opening consisting of one round of dilation and erosion, and closing consisting of one round of erosion and dilation, is sufficient. When the size of the defects increases, more rounds of dilation or erosion are required; here, detail on the boundary of products may be lost. Therefore, if the size of the defects in images after thresholding-based segmentation is relatively large, an alternative thresholding technique rather than post-processing should be adopted.


(a)

(b) Figure 2.6 Image-opening and -closing for defects removal of the segmented image in Figure 2.3: (a) opening with 3 rounds of erosion and dilation; (b) closing with 2 rounds of dilation and erosion.

3.2 Region-based segmentation There are two region-based segmentation techniques: growing-and-merging (GM), and splitting-and-merging (SM) (Navon et al., 2005). In the GM methods, a pixel is initially selected as a growing region. Pixels neighboring the region are iteratively merged into the region, if the pixels have similar characteristics (e.g. intensity and texture) to the region concerned, until no more pixels can be merged. Afterwards, the growing procedure is repeated with another pixel that has not been merged into any regions, until all the pixels in the image have been merged into various regions. It usually happens that images are over-segmented, which means that there are some regions that are too small to remain as independent regions, mostly due to the presence of noise. Therefore, post-processing is generally conducted to merge the over-segmented regions into their nearby independent regions of larger area. In the SM methods, the whole image is initially regarded as a big region, and is split iteratively into smaller regions with uniform image characteristics (e.g. color, gradient, and texture). The segmentation is terminated when there are no longer any regions with un-uniform characteristics to be split. Similarly to GM, to overcome the problem of


over-segmentation, very small regions are merged into neighboring regions that are large enough to be independent regions. Region-based segmentation methods are usually proposed for the purpose of segmenting complex images in which the number of classes is large and unknown. However, in the segmentation of food images, the number of classes is normally already assigned as two – i.e. food products and background, or defect and non-defect. Further to this, region-based techniques are usually time-consuming. Therefore, region-based methods are less popular in the applications of computer vision in the food industry. One of the limited instances of the use of a region-based method is a stick growingand-merging algorithm proposed by Sun and Du (2004) mostly for the segmentation of pizza toppings; it is impossible to segment these by using thresholding-based methods.

3.3 Gradient-based segmentation Computing the image gradient is favored simply because boundaries of local contrast can be effortlessly observed in the gradient images, and thus the edges of objects can also be easily detected. Image segmentation is meanwhile accomplished, since the edges of objects in images are located. Therefore, gradient-based segmentation is also called “edge detection.” Typically, in gradient-based segmentation, the gradient of an image is computed using convolute gradient operators, and a threshold t is set to distinguish effective edges whose gradient is larger than t. The threshold can usually be selected from the cumulative of the gradient histogram of images, working on the scheme that 5–10 percent of pixels with the largest gradient can be chosen as edges (Jain, 1989). 3.3.1 Gradient operator

Considering the image as a function f of the intensity value of pixels (x, y), the gradient g can be computed by: 2 ∂f 2 ∂f (2.6) + g= ∂x ∂y In digital images, a gradient operator is similar to an averaging operator (for noise removal), which is a weighted convolution operator utilizing the neighboring pixels for the operation. However, unlike the averaging operator, the weightings of a gradient operator are not exclusively positive integers. Indeed, at least one negative integer is present in the weighting so that the intensity value of the central pixel can be subtracted from the values of the neighboring pixels, in order to increase the contrast among adjacent pixels for computing gradients. Gradients can be computed in a total of eight directions (see Figure 2.7). Further to this, the sum weight of a gradient operator is usually 0. Some of the well-known gradient operators that have been widely used are the Sobel, Prewitt, Roberts, and Kirsch operators (Russ, 1999). 3.3.2 Laplace operator Although most of the operators described above are competent when the intensity transition in images is very abrupt, as the intensity transition range gradually gets


Figure 2.7

Eight possible directions in which to compute the gradient.

wider and wider the gradient operators might not be as effective as they are supposed to be. Consequently, the second-order derivative operators depicted below might be considered as alternative approaches for the gradient operators: ∇ 2f =

∂2 f ∂2 f + ∂x 2 ∂y 2

(2.7)

Similarly, the second-order derivative operators are also convolute operators in digital images. The following is one of the widely used derivative operators, the Laplace operator, in which the second-order derivative is determined by subtracting intensity values of the neighboring pixels from the value of the central pixel: −1

−1

−1

−1

+8

−1

−1

−1

−1

However, the Laplace operator is very sensitive to noise, and thus it is not rated as a good edge detector. Instead, some generalized Laplace operators might be used, such as the approximation of the Laplacian of Gaussian function, which is a powerful zerocrossing detector for edge detection (Marr and Hildreth, 1980). To our knowledge these operators have not yet been employed in the food industry, so they are not discussed further here. 3.3.3 Other techniques The first quantitative measurements of the performance of edge detectors, including the assessment of the optimal signal-to-noise ratio and the optimal locality, and the maximum suppression of false response, were performed by Canny (1986), who also proposed an edge detector taking into account all three of these measurements. The Canny edge detector was used in the food industry for boundary extraction of food products (Du and Sun, 2004; 2006b; Jia et al., 1996). Another popular gradient-based technique is the active contour model (ACM), otherwise known as “Snakes,” which transforms the problem of edge detection into an


energy optimization problem. An active and deformable contour of the object is first defined and then, step-by-step, the active contour is moved towards the real object contour by minimizing the energy. The primary disadvantage of the ACM is that the initial contour sometimes cannot be close enough to the object edge, causing failure of convergence of the active contour with the object edge. Fortunately, this problem can be solved by the gradient vector flow (GVF), which can overcome the defect of the traditional external flow and move the active contour towards the desired object edge more efficiently. So far, the ACM method has been proposed for the segmentation of touching, adjacent rice kernels (Wang and Chou, 1996). However, the technical details of the ACM and GVF are far beyond our discussion here. Readers interested in these techniques can refer to the original work on ACM and GVF by Kass et al. (1988) and Xu and Prince (1998), respectively.

3.4 Classification-based segmentation Classification-based segmentation is the second most popular method, after thresholding-based segmentation, used in the food industry. Classification-based segmentation is a pixel-orientated method in which each pixel is regarded as being an independent observer whose variables are generally obtained by image features (e.g. color, shape, and texture). Afterwards, a matrix that contains every pixel as an observer is obtained as the input of the classification. Each observer is then classified (object and background, or defect and non-defect, etc.) according to its variables, using a learning model (Du and Sun, 2006c). Normally, a set of images that is successfully segmented manually using human vision is provided as the training set (called supervision learning) in the classification. Coefficients of the learning model are obtained so that the testing image set can be classified using the same model with the acquired coefficients. An example of the supervised classification procedure is illustrated in Figure 2.8. Although having the training image set is an advantage, it is not absolutely necessary because there are some unsupervised learning techniques available, such as clustering and the self-organizing-map, by which the observers can be clustered into different classes without any other a priori knowledge. Nevertheless, this unsupervised training is not as accurate as supervised in most cases; therefore, it is still preferable to use the training image set (supervision) if possible. One drawback of the classification-based methods compared with gradient-based and region-based techniques is that the goal of the segmentation needs to be known prior to carrying out segmentation – in other words, the number of classes that the images can be segmented into should be given. For instance, in the segmentation of a food product from the background, segmenting into two classes (i.e. object and background) is the segmentation goal; in defect detection in apples, the goal of segmentation is defect and non-defect. Fortunately, in most segmentation cases in the food industry the goal of segmentation is mostly known beforehand. Therefore, classification-based segmentation is widely used in the food industry. Another drawback of this technique is that its performance is subject to two major factors, i.e. the features obtained from images as variables of the observers and the learning models used.


Training Color Texture . . .

A / B /. . .

Color Texture . . .

A / B /. . .

Color Texture . . . ...

...

...

Color Texture

Learning model with unknown coefficients

Color Texture . . .

A / B /. . . ... A / B /. . . A / B /. . .

Coefficients

Testing

Color Texture . . .

A / B /...

Color Texture . . .

A / B /...

Color Texture . . . ...

...

...

Color Texture . . . Color Texture . . . Figure 2.8

Learning model with obtained coefficients

A / B /... ... A / B /... A / B / ...

Classification-based segmentation.

3.4.1 Features extraction

Since pixel intensity value is the primary information stored within pixels, it is the most popular and important feature used for classification. The intensity value for each pixel is a single value for a gray-level image, or three values for a color image. An alternative approach to the acquisition of intensity values from a single image is the multispectral imaging technique, with which more than one image of the same product at the same location can be obtained at different wavelengths. Afterwards, intensity values of the same pixel are acquired from the various images as the classification features of pixels. This technique has drawn strong interest from researchers carrying out work in applequality evaluation using computer vision technology (Leemans et al., 1999; Blasco et al., 2003; Kleynen et al., 2005). Sometimes, to acquire more information about the pixels, its features can be extracted from a small region that is centered on the pixel. Therefore, besides the intensity value, the image texture – which is an important factor of the product surface for pattern recognition due to its powerful discrimination ability (Amadasun and King, 1989) – can also be extracted as a classification feature of pixels. For further technical information on the extraction of image texture features, refer to the review by Zheng et al. (2006). 3.4.2 Classification methods

3.4.2.1 Dimension reduction Since a large amount of data is present in the input matrix for classification, it is generally preferred that the dimension of the original matrix is reduced before classification. Although principal component analysis (PCA) is a powerful


dimension-reduction method, it is mostly used for the purpose of reducing classification variables. Consequently, PCA is not suitable for classification-based segmentation because classification-based segmentation demands a reduction in the number of classification observers. Accordingly, the self-organizing map (SOM) has been developed. The SOM, generalized by extracting the intrinsic topological structure of the input matrix from the regularizations and correlations among observers, is an unsupervised neural network in which each neuron represents a group of observers with similar variables. Afterwards, the SOM can be used for classification rather than the original observers, and the observers are assigned to the class of the neuron that the observers belong to (Chtioui et al., 2003; Marique et al., 2005). 3.4.2.2 Classification Although there are several different types of techniques available at this stage – i.e. statistical technique (ST), neural network (NN), support vector machine (SVM), and fuzzy logic (FL) – only the Bayesian theory (a ST method) and fuzzy clustering (combination of ST and FL) have been proposed in the food industry so far. The Bayesian theory generates the Bayesian probability P(Ci |X ) for a pixel (observer) to belong to the class Ci by its features (variables) X using the following equation: P(C i |X ) =

P(X |C i )P(C i ) P(X )

(2.8)

where P(X |Ci ) is the probability of an observer belonging to Ci having the variable X ; P(Ci ) is a priori the probability of classifying an observer into class Ci ; and P(X ) is the a priori probability of an observer having the variable X . Later, a threshold on the Bayesian probability is selected, and if the probability of an observer is larger than the threshold, the observer is classified into the class Ci . Fuzzy clustering is a combination of a conventional k-mean clustering and a fuzzy logic system in order to simulate the experience of complex human decisions and uncertain information (Chtioui et al., 2003; Du and Sun, 2006c). In fuzzy clustering, each observer is assigned a fuzzy membership value for a class, and an objective function is then developed based on the fuzzy membership value. The objective function will be minimized iteratively, until convergence is reached, by updating the new fuzzy membership value according to the observers and the number of iterations. The criterion determining the convergence of the objective function is generally defined as when the difference of the values of the objective function between two successive iterations is significantly small.

3.5 Other segmentation techniques 3.5.1 Watershed

The concept of watersheds, which are introduced into digital images for morphological processing, originally comes from topography. In morphological processing, images are represented as topographical surfaces on which the elevation of each point is assigned as the intensity value of the corresponding pixel. Before the detection of watersheds in images, two concepts (i.e. the minimum and catchment basin) need to


be defined. The minimum is a set of connected pixels with the lowest intensity value in images, while catchment basin, covering the minimum, is another set of pixels in which water only flows across pixels to the minimum inside (Vincent and Soille, 1991). While flooding water from the minimum of a catchment basin occurs gradually, dams corresponding to watersheds are built surrounding the catchment basin to prevent water from falling into another catchment basin. Accordingly, regions are formed using the watersheds, and image segmentation can be accomplished simultaneously. The watersheds can be constructed from different scales of images – grayscale (Vicent and Soille, 1991), binary (Casasent et al., 2001), and gradient (Du and Sun, 2006a). Owing to the presence of noise and local irregularities, there are far more minima from which far more catchment basins are formed, causing the over-segmentation of images. To overcome this problem, algorithms are designed. One method for preventing over-segmentation is to eliminate the undesired minima, using morphological operators such as opening and closing. One such method was proposed by Du and Sun (2006a) to segment pores in pork ham images. In other methods, post-processing is conducted to merge the over-segmented regions with similar image characteristics together again. Such a method with a graphic algorithm to determine the similarity of merging neighboring regions was developed by Navon et al. (2005). 3.5.2 Hybrid-based segmentation Although a large number of segmentation techniques have been developed to date, no universal method can perform with the ideal efficiency and accuracy across the infinity diversity of imagery (Bhanu et al., 1995). Therefore, it is expected that several techniques will need to be combined in order to improve the segmentation results and increase the adaptability of the methods. For instance, Hatem and Tan (2003) developed an algorithm with an accuracy of 83 percent for the segmentation of cartilage and bone in images of vertebrae by using the thresholding-based method twice. First the images were segmented by a simple threshold, and regions of cartilage and bones were formed. Subsequently, another two thresholds – one based on size and the other on elongation – were used to filter the segmented cartilage and bone regions, but not the real cartilage or bone. Although classification-based segmentation yields better segmentation results than thresholding-based methods in the segmentation of longissimus dorsi beef images, the computation speed is strongly affected by using classificationbased methods. Therefore, a classification-based method was first employed in a study by Subbiah et al. (2004) to successfully segment the longissimus dorsi in a set of images from which an ideal threshold for histogram thresholding was automatically computed and used to segment the results of images. This algorithm retained the accuracy of the classification-based segmentation (being only 0.04 percent slightly lower), and meanwhile reduced the computation time by 40 percent.

4 Conclusions Owing to the imperfections of image acquisition systems, image pre-processing such as image filtering and histogram manipulation is performed to remove noise and

Nomenclature 53

enhance contrast for the purpose of facilitating subsequent processing. Later, image segmentation is conducted to discriminate food products from the background for further analysis. Thresholding-based segmentation segments images by their histograms using an optimal threshold that can be chosen by manual selection, isodata algorithm, objective functions, clustering, and many other techniques. Image-closing and -opening are sometimes employed to correct the segmentation errors produced by thresholding. In region-based segmentation, two schemes might be considered – growing-andmerging, and splitting-and-merging. Gradient-based segmentation, also known as edge detection, is segmenting images by detecting the edges of objects, utilizing gradient operators, derivative operators, and active contour models. In classification-based segmentation, pixels are allocated to different classes (e.g. objects and background) by features such as intensity and texture. Other techniques, such as the use of watersheds, have also been developed. Despite this, because image segmentation is by nature still an ill-defined problem, none of the methods described can perform ideally across diverse images. It has been suggested recently that several techniques might be combined together for the sake of improving the segmentation result and simultaneously increasing segmentation speed.

Nomenclature ∇ µ σ ω C e f f g H h i, j L l M m O P t w X x, y

derivative first-order cumulative between-class variance zeroth-order cumulative class partition error image transformed image gradient histogram normalized histogram parameters maximum intensity minimum intensity size of image filters average intensity calculation probability threshold weight of image filters variable coordinates


Abbreviations: FL fuzzy logic NN neural networks PCA principal component analysis SOM self-organizing map ST statistical learning SVM support vector machines

References Abutaleb AS (1989) Automatic thresholding of grey-level pictures using two-dimensional entropies. Pattern Recognition, 47 (1), 22–32. Amadasun M, King R (1989) Textural features corresponding to textural properties. IEEE Transactions on Systems, Man, and Cybernetics, 19 (5), 1264–1274. Bhanu B, Lee S, Ming J (1995) Adaptive image segmentation using a genetic algorithm. IEEE Transactions on Systems, Man, and Cybernetics, 25 (12), 1543–1567. Blasco J, Aleixos N, Moltó E (2003) Machine vision system for automatic quality grading of fruit. Biosystems Engineering, 85 (4), 415–423. Canny J (1986) A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8 (6), 679–698. Casasent D, Talukder A, Keagy P, Schatzki T (2001) Detection and segmentation of items in X-ray imagery. Transactions of the ASAE, 44 (2), 337–345. Chtioui Y, Panigrahi S, Backer LF (2003) Self-organizing map combined with a fuzzy clustering for color image segmentation. Transactions of the ASAE, 46 (3), 831–838. Du C-J, Sun D-W (2004) Shape extraction and classification of pizza base using computer vision. Journal of Food Engineering, 64 (4), 489–496. Du C-J, Sun D-W (2006a) Automatic measurement of pores and porosity in pork ham and their correlations with processing time, water content and texture. Meat Science, 72 (2), 294–302. Du C-J, Sun D-W (2006b) Estimating the surface area and volume of ellipsoidal ham using computer vision. Journal of Food Engineering, 73 (3), 260–268. Du C-J, Sun D-W (2006c) Learning techniques used in computer vision for food quality evaluation: a review. Journal of Food Engineering, 72 (1), 39–55. Faucitano L, Huff P, Teuscher F, Cariepy C, Wegner J (2005) Application of computer image analysis to measure pork marbling characteristics. Meat Science, 69 (3), 537–543. Gauch JM (1992) Investigations of image contrast space defined by variations on histogram equalization. CVGIP: Graphical Models and Image Processing, 54 (4), 269–280. Hatem I, Tan J (2003) Cartilage and bone segmentation in vertebra images. Transactions of the ASAE, 46 (5), 1429–1434. Hwang H, Park B, Nguyen M, Chen Y-R (1997) Hybrid image processing for robust extraction of lean tissue on beef cut surface. Computers and Electronics in Agriculture, 17 (3), 281–294.

References 55

Jain AK (1989) Fundamentals of Digital Image Processing. Englewood Cliffs: Prentice-Hall. Kapur JN, Saho PK, Wong AKC (1985) A new method for gray level picture thresholding using the entropy of the histogram. ComputerVision, Graphics, and Image Processing, 29, 273–285. Kass M, Witkin A, Terzoulos D (1988) Snake: active contour models. International Journal of Computer Vision, 1 (4), 321–331. Kittler J, Illingworth J (1985) On threshold selection using clustering criteria. IEEE Transactions on Systems, Man, and Cybernetics, 15 (5), 652–665. Kittler J, Illingworth J (1986) Minimum error thresholding. Pattern Recognition, 19 (1), 41–47. Kleynen O, Leemans V, Destain M-F (2005) Development of a multi-spectral vision system for the detection of defects on apples. Journal of Food Engineering, 69 (1), 41–49. Leemans V, Magein H, Destein M-F (1999) Defect segmentation on ‘Jonagold’ apples using color vision and a Bayesian classification method. Computers and Electronics in Agriculture, 23 (1), 43–53. Marique T, Pennincx S, Kharoubi A (2005) Image segmentation and bruise identification on potatoes using a Kohonen’s self-organizing map. Journal of Food Science, 70 (7), E415–E417. Marr D, Hildreth E (1980) Theory of edge detection. Proceedings of the Royal Society of London B, 207, 187–217. Navon E, Miller O, Averbuch A (2005) Image segmentation based on adaptive local thresholds. Image and Vision Computing, 23 (1), 69–85. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9 (1), 62–66. Pun T (1980) A new method for gray-level picture thresholding using the entropy of the histogram. Signal Processing, 2 (3), 223–237. Read W (1982) Comments on two papers in pattern recognition. IEEE Transactions on System, Man, and Cybernetics, 12, 429–430. Ridler TW, Calvard S (1978) Picture thresholding using an iterative selection method. IEEE Transactions on Systems, Man, and Cybernetics, 8 (8), 630–532. Russ J C (1999) The Image Processing Handbook, 3rd edn. Boca Raton: CRC Press. Sahoo P, Wilkins C, Yeager J (1997) Threshold selection using Renyi’s entropy. Pattern Recognition, 30 (1), 71–84. Subbiah J, Ray N, Kranzler GA, Acton ST (2004) Computer vision segmentation of the longissimus dorsi for beef quality grading. Transactions of the ASAE, 47 (4), 1261–1268. Sun D-W (2000) Inspecting pizza topping percentage and distribution by a computer vision method. Journal of Food Engineering, 44 (4), 245–249. Sun D-W, Du C-J (2004) Segmentation of complex food images by stick growing and merging algorithm. Journal of Food Engineering, 61 (1), 17–26. Tobias OJ, Seara R (2002) Image segmentation by histogram thresholding using fuzzy sets. IEEE Transactions on Image Processing, 11 (12), 1457–1465. Tsai WH (1985) Moment-preserving thresholding: a new approach. Computer Vision, Graphics, and Image Processing, 29 (3), 377–393.


Vincent L, Soille P (1991) Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13 (6), 583–598. Wang Y-C, Chou J-J (1996) Automatic segmentation of touching rice kernels with an active contour model. Transactions of the ASAE, 47 (5), 1803–1811. Xu C, Prince JL (1998) Snakes, shapes, and gradient vector flow. IEEE Transactions on Image Processing, 7 (3), 359–363. Zheng C, Sun D-W, Zheng L (2006) Recent applications of image texture for evaluation of food qualities – a review. Trends in Food Science & Technology, 17 (3), 113–128.

Object Measurement Methods Chaoxin Zheng and Da-Wen Sun Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland

1 Introduction After image segmentation, where objects are discriminated from the background, the characteristics of objects, known as object measurements, are calculated. These measurements are the core elements in a computer vision system, because they contain useful information for image understanding and interpretation, and object classification (Ballard and Brown, 1982). In the food industry, these object measurements carry the direct information that can be used for quality evaluation and inspection. Unsuccessful extraction of the proper object measurements would probably result in the failure of the computer vision system for food quality inspection. In computers, images are stored and processed in the form of matrices. Elements of the matrices are referred to as pixels, in which two types of information are presented – geometric information (i.e. the location of pixels in images) and surface information (the intensity values associated with pixels). From the geometric information, two different object measurements can be obtained: size and shape. From the surface information, color and texture can be extracted. These four measurements – size, shape, color, and texture – are rated as the primary types of object measurements that can be acquired from any images (Du and Sun, 2004a). A great number of methods have been developed for the acquisition of object measurements, including size, shape, color, and texture, over the past few decades. Even so, there is not yet a perfect method for each type of measurement, and especially for texture measurements. This is because of the lack of a formal and scientific definition of image texture while facing the infinite diversity of texture patterns (Zheng et al., 2006a). There are some problems with the methods that cause them not to work properly under certain circumstances. For example, Fourier transform, which is a potential method for extracting shape measurements, will not work properly when there is re-entrant on the boundary of objects (Russ, 1999). Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


3

58 Object Measurement Methods

The objective of this chapter is to review the current methods available for the extraction of object measurements. The advantages and disadvantages of most methods are also discussed in order to provide those researchers in the food industry who intend to pursue computer vision for quality evaluation with some guidelines on choosing effective object measurement methods.

2 Size Since three-dimensional (3-D) information regarding objects is lost during image acquisition unless special techniques such as structural lighting are used (Baxes, 1994), size measurements of objects in digital images are restricted to being one-dimensional (1-D) and two-dimensional (2-D). The measurements of volume and surface area, which are 3-D measurements, are thus less popular. Length, width, area, and perimeter are the preferred measurements, and especially the latter two. The area and perimeter are calculated simply by counting the number of pixels belonging to an object, and summing the distance between every two neighboring pixels on the boundary of the object, respectively. No matter how irregular the shape of the object, or what its orientation is, measurements of area and perimeter are stable and efficient once the object has been successfully segmented from the background. Calculation of the length and width is much more complex than that of area and perimeter, especially for food objects, which generally have very irregular shapes. Nevertheless, some measurements for length and width have been developed by researchers and are used in the food industry. The measurements most commonly used are Feret’s Diameter, the major axis, and the minor axis (Zheng et al., 2006b). Feret’s Diameter is defined as the difference between the largest and the smallest of the coordinates of an object at different rotations (Figure 3.1). The major axis is the longest line that can be drawn across the object, and is obtained by examining the distance between every two boundary pixels and taking the longest. The minor axis, defined as the longest line

x FD

FD

0 Figure 3.1

Illustration of Feret’s Diameter.

y

Shape 59

that can be drawn across the object perpendicular to the major axis, can therefore be determined after determining the major axis. Further to these, the major and minor axes can also be defined as those in an ellipse which is fit to the object using ellipse-fitting methods (Russ, 1999; Mulchrone and Choudhury, 2004; Zheng et al., 2006c). One drawback to length and width measurements is that the orientation at which the length and width are measured must be determined prior to the calculation. Since the shape of food products generally changes during processing, the orientation at which the length and width are calculated needs constantly to be updated. Far more calculations will thus be required, and this is undesirable for on-line food quality evaluation. Consequently, area and perimeter measurements are preferable to length and width measurements for the evaluation of the size of products such as tomatoes (Tu et al., 2000; Devaux et al., 2005), pork (Collewet et al., 2005; Faucitano et al., 2005), and grains (Srikaeo et al., 2006).

3 Shape Shape, as with size, is another geometric measurement of food products. Furthermore, shape plays an important part in the purchase decision of customers (Leemans and Destain, 2004), and this establishes the significance of shape measurement in the applications of computer vision for food quality inspection. Typical applications of shape include the evaluation of product acceptance to customers, using machine learning techniques (Du and Sun, 2004a, 2006; Leemans and Destain, 2004), and the discrimination of products with different characteristics (Ghazanfari and Irudayaraj; 1996; Zion et al., 1999; 2000). An example illustrating pizza bases with different shapes is shown in Figure 3.2 (Du and Sun, 2004b). Along with these applications, many methods have been developed to characterize product shapes, including two major categories – size-dependent measurements and size-independent measurements.

(a)

(b)

(c)

(d)

Figure 3.2 Pizza bases of different shapes (Du and Sun, 2004b): (a) flowing; (b) poor alignment; (c) poor processing; (d) standard.


3.1 Size-dependent measurements Size-dependent measurements (SDM) are descriptors of shape. These descriptors are formed by the proper combinations of size measurements. The SDM that have been applied in the food industry include (Zheng et al., 2006b): 1. 2. 3. 4.

Compactness, which is the ratio of area over the square perimeter Elongation, which is the ratio of the major axis over the minor axis Convexity, which is the ratio of the convex perimeter over the perimeter Roughness, which is the ratio of area over the square major axis.

It can be seen that the definitions of these SDM are easy to understand, and that their calculation is also straightforward. Compactness provides a good example of how to describe shape by using SDM. For a perfectly circular food product, the largest value of compactness, 1, is reached. Variations of the shape, as more and more corners are added to the product, will gradually reduce the value of compactness.

3.2 Size-independent measurements The ideal measurement of shape is that which can be used to discriminate one shape adequately from another. In other words, with this ideal measurement, every shape has a unique value (Russ, 1999). It is thus a matter of concern that size-dependent measurements (SDM) may be insufficient to characterize the shape of every food product because of the great irregularities of shape – consider, for example, a head of broccoli, and the entire body of a fish. The chance of two different, very irregular shapes having the same value under these simple combinations of size measurements is still very large. Size-independent measurements (SIM), including region-based and boundary-based methods, have consequently been developed. 3.2.1 Region-based method The region-based method, also known as the spatial moment, is based on the statistical characteristics of object regions. As pixels are the basic elements forming an object region in digital images, the spatial moment consists of the statistics regarding the spatial information of all pixels inside the object (Jain, 1989). The most basic measurement by spatial moment is the centre of mass (x, y), which can be calculated by the following equations: 1 x = x (3.1) N x y

y =

1 y N x y

(3.2)

where N is the total number of pixels inside the object, and (x, y) is the coordinates of a pixel. The (p, q) order of the central moment can thus be obtained by: Mpq = (x − x)p (y − y)q (3.3) x

y

Shape 61

Actually, the spatial moment measures the properties of an object rather than those of its shape. It is an effective method for the purpose of discriminating one shape from another, whereas its function in describing the changes of object shapes vividly is limited (Zheng et al., 2006b). Applications of the spatial moment can be found in the classification of fish specifies (Zion et al., 1999, 2000), where the Fourier transform cannot work properly due to the re-entrants presented on the boundary of fish bodies. 3.2.2 Boundary-based method

In contrast to the region-based method, the boundary-based method obtains shape measurements by first representing the boundary with the spatial information of boundary pixels, and then analyzing and extracting measurements from the spatial information. 3.2.2.1 Boundary representation The simplest way to represent an object boundary is by extracting and storing the coordinates (x, y) of every pixel on the boundary in a vector. Another method of boundary representation is called the chain code. In this method, eight directions of a pixel are defined. Since the boundary is constituted by connected pixels, a single pixel is selected and the directions of subsequent pixels are stored in the chain code one by one until finally the initial pixel is reached. Furthermore, the radius from every pixel on the boundary to the center of object can be used for boundary representation (Figure 3.3), and thus another method has been developed in which the radiuses are described as a function of their angles by the following equation: r = f (θ)

(3.4)

Although the boundary can be represented or effectively reconstructed with the methods described above, these representations are too sensitive to the size and orientation of objects, and thus are not directly used as shape measurements (Baxes, 1994). Instead, Fourier transform and autoregressive models, sometimes combined with principal component analysis, are usually applied to extract the shape measurements from the vector, chain code, and radius function, so that effects arising from the size or orientation of the object can be eliminated. r r u

0 (a)

2p

u

(b)

Figure 3.3 Representing an object’s boundary by using radius function: (a) the boundary of a beef joint; (b) the radius function of (a).


3.2.2.2 Boundary analysis and classification Fourier transform Fourier transform (FT) reconstructs the boundary representation, in most cases the radius function, into a summation of a series of cosine and sine terms at increasing frequency, as in equation 3.5: 1 f (θ)e−i2πvθ/N N N

F (v) =

(3.5)

θ=0

where u is the coefficient of the FT, and N is the total number of frequencies. The coefficients are further used for food quality evaluation in two different ways. In the first approach, principal component analysis is applied to all the coefficients in order to compress the data dimension by selecting the first few principal components containing the significant information about object shape. The selected principal components are later employed to classify or to predict the food of, for example, such things as pizza bases (Du and Sun, 2004b) and apples (Currie et al., 2000). In the second approach, the absolute value of each coefficient is summed up as shape measurements. Such an application has been set up by Zheng et al. (2006d) to predict the shrinkage of large, cooked beef joints as affected by water-immersion cooking. The advantages of FT are perceivable. Using the Fourier coefficients rather than the original radius can eliminate the effects of the location, size, and orientation of objects on the shape measurements (Schwarcz and Shane, 1969), which is difficult to achieve by using other methods. However, the method of FT has one disadvantage. For object shape with re-entrants, the boundary function described in equation (3.4) has multiple values at the same entry, which will therefore cause a failure in constructing the Fourier series. Although the problem can be solved by the integration of another parameter into the radius function (Russ, 1999), a far greater computation load will also be experienced. FT is consequently only preferred for the description of shapes without re-entrants, in the food industry. Autoregressive models The measurements obtained from Fourier transform are useful for the classification of different shapes. For the purpose of extracting the global measurements or the similar characteristics of a group of shapes, autoregressive models can be used (Kashyap and Chellappa, 1981). This method is described by the following equations:

ux ( j) =

N

ax (k)ux ( j − k) + εx ( j)

(3.6)

k=1

x( j) = ux ( j) + µx uy ( j) =

N

ay (k)uy ( j − k) + εy ( j)

(3.7) (3.8)

k=1

y( j) = uy ( j) + µy

(3.9)

where u is the zero mean stationary random sequence (Jain, 1989), n is the nth pixel on the boundary, ε is the uncorrelated sequence with zero mean and a specified variance,

Color 63

and µ is the ensemble mean (Jain, 1989) of x( j) and y( j), which are the x and y coordinates, respectively, of pixel j. In equations (3.6)–(3.9), the values of a and ε are specific to each shape, and are therefore considered to be shape measurements.

4 Color Color provides the basic information for human perception. Further to this, color is also elementary information that is stored in pixels to constitute a digital image. Color is hence rated as one of the most important object measurements for image understanding and object description. According to the tri-chromatic theory, that color can be discriminated by the combination of three elementary color components (Young, 1802; MacAdam, 1970), three digital values are assigned to every pixel of a color image. Two typical statistical measurements, including the mean and variance, are obtained from each component as color measurements. Different types of values stored for the three color components, and different color reproduction methods using these three values, lead to different color spaces. These spaces can be generally classified into three types: hardware-orientated, human-orientated, and instrumental. The measurements of color are dependent on these spaces.

4.1 Hardware-orientated spaces Hardware-orientated spaces are developed in order to facilitate hardware processing, such as capturing, storing, and displaying. The most popular hardware-orientated space is the RGB (red, green, blue) space, so-called because this is the way in which cameras sense natural scenes and display phosphors work (Russ, 1999). RGB is consequently used in most computers for image acquisition, storage, and display. Color in the RGB space is defined by coordinates on three axes, i.e. red, green, and blue, as illustrated in Figure 3.4. Apart from RGB, another popular hardware-orientated space is the YIQ Blue

Red

Green Figure 3.4 Illustration of the RGB color space.


(luminance, in-phase, quadrature) space, which is mainly used for television transmission. RGB space is transformed into YIQ space by using equation (3.10) to separate the luminance and the chrominance information in order to facilitate compression applications (Katsumata and Matsuyama, 2005). ⎤⎡ ⎤ ⎡ ⎤ ⎡ Yˆ 0.299 0.587 0.114 Rˆ ⎣ Î ⎦ = ⎣0.596 −0.275 −0.321⎦⎣Gˆ ⎦ (3.10) ˆ 0.207 −0.497 0.290 Bˆ Q As well as YIQ space, YUV, YCbCr, and YCC spaces are also used in color transmission; the principles are similar to that of YIQ, and thus they are not further discussed here. CMYK (cyan, magenta, yellow, black) is also a hardware-orientated color space. However, CMYK is mainly employed in printing and copying output, and hence is not used for color measurements in the food industry. By combining values from each component in the hardware-orientated spaces, color can be effectively measured. Even a very small variation in color can be sensed. The hardware-orientated spaces are therefore popular in evaluating color changes of food products during processing. For instance, small variations of color measurements obtained from the RGB space can be used to describe changes of temperature and time during the storage of tomatoes (Lana et al., 2005). Nevertheless, hardware-orientated spaces are non-linear with regard to the visual perception of human eyes, and consequently are not capable of evaluating the sensory properties of food products. In order to achieve this, human-orientated color spaces are used.

4.2 Human-orientated spaces Human-orientated spaces, which include HSI (hue, saturation, intensity), HSV (hue, saturation, value), and HSL (hue, saturation, lightness), have been developed with the aim of corresponding to the concepts of tint, shade, and tone, which are defined by an artist based on the intuitive color characteristics. Hue is measured by the distance of the current color position from the red axis, which is manifested by the difference in color wavelengths (Jain, 1989). Saturation is a measurement of the amount of color – i.e. the amount of white light that is present in the monochromatic light (Jain, 1989; Russ, 1999). The last component – intensity, value, or lightness – refers to the brightness or luminance, defined as the radiant intensity per unit projected-area by the spectral sensitivity associated with the brightness sensation of human vision (Hanbury, 2002). Compared with RGB space, which is defined by cuboidal coordinates, the coordinates used to define color in HSI, HSV, and HSL are cylindrical (see Figure 3.5). The relationship between the RGB space and the HSI space can be described by the following equations:

⎧

⎪ π 2Rˆ − Gˆ − Bˆ ⎪ −1 ˆ ⎪ + π /2π if (Gˆ < B) √ ⎪ ⎨ 2 − tan ˆ 3(Gˆ − B) ˆ =

H (3.11) ⎪ ˆ ˆ ˆ π 2 R − G − B ⎪ ⎪ −1 ˆ ˆ ⎪ /2π if (G > B) √ ⎩ 2 − tan ˆ 3(Gˆ − B)

Color 65

Green

Red Hue Saturation Blue

Intensity

Figure 3.5 Illustration of the HSI color space.

Sˆ = 1 − Î =

ˆ G, ˆ B) ˆ min(R, Î

Rˆ + Gˆ + Bˆ 3

(3.12)

(3.13)

As specified above, HSI space has been developed by considering the concept of visual perception in human eyes; color measurements obtained from HSI are thus better related to the visual significance of food surfaces. There is therefore greater correlation between the color measurements from human-orientated spaces and the sensory scores of food products. This has been clarified by a study in which color measurements from the HSV space were found to give a better performance than those from the RGB space in the evaluation of acceptance of pizza toppings (Du and Sun, 2005). However, the defect of human-orientated spaces is that they, as with human vision, are not sensitive to a small amount of color variation. Therefore, human-orientated color spaces are not suitable for evaluating changes of product color during processing.

4.3 Instrumental spaces Instrumental spaces are developed for color instruments, such as the colorimeter and colorimetric spectrophotometer. Many of these spaces are standardized by the CIE (Commission International de L’Éclairage) under the specifications of lighting source, observer, and methodology spectra (Rossel et al., 2006). The earliest such space is the one named XYZ, where Y represents the lightness while X and Z are two primary


virtual components (Wyszecki and Stiles, 1982). Equation (3.14) can be used to convert color measurements linearly from RGB space to XYZ space. ⎤⎡ ⎤ ⎡ ⎤ ⎡ Xˆ 0.412453 0.357580 0.180423 Rˆ ⎣Yˆ ⎦ = ⎣0.212671 0.715160 0.072169⎦⎣Gˆ ⎦ (3.14) 0.019334 0.119194 0.950227 Bˆ Zˆ Although it is useful in defining color, XYZ is not ideal for the description of color perception in human vision. CIE La∗ b∗ and CIE Lu∗ v∗ color spaces, which are the non-linear transformation of XYZ as described below, are thus brought out and adopted in many color measuring instruments. 116 × (Yˆ /Y )1/3 − 16 if (Yˆ /Y ) > 0.008856 (3.15) Lˆ = 903.3 × (Yˆ /Y ) else a∗ = 500[(Xˆ /X )1/3 − (Yˆ /Y )1/3 ]

(3.16)

ˆ )1/3 ] b∗ = 200[(Yˆ /Y )1/3 − (Z/Z

(3.17)

u∗ = 13 × Lˆ × (u − u )

(3.18)

v ∗ = 13 × Lˆ × (v − v )

(3.19)

where X , Y , and Z are the values corresponding to the standardized point D65 shown below: ⎡ ⎤ ⎡ ⎤ X 95.047 ⎣Y ⎦ = ⎣ 100 ⎦ (3.20) Z 108.883 Here, u , u , v , and v are determined by equations (3.21)–(3.24), respectively: u =

4Xˆ Xˆ + 15Yˆ + 3Zˆ

(3.21)

u =

4X X + 15Y + 3Z

(3.22)

v =

9Yˆ Xˆ + 15Yˆ + 3Zˆ

(3.23)

v =

9Y X + 15Y + 3Z

(3.24)

The color component L is referred to as the lightness or luminance, while a∗ (u∗ ) is defined along the axis of red–green, and b∗ (v∗ ) is defined along the axis of yellow– blue. A positive value of a∗ (u∗ ) indicates that red is the dominant color, while a negative value suggests the dominance of green. The same applies the b∗ (v∗ ) component on the yellow–blue axis – a positive value indicates that yellow is dominant, while a negative value suggests the dominance of blue (Russ, 1999).

Texture 67

Since color measured by computer vision can be easily compared to that obtained from instruments, these instrumental color spaces offer a possible way of evaluating the performance of computer vision systems in measuring object color. Such an application was previously established by O’Sullivan et al. (2003) for the grading of pork color.

5 Texture Starting in the 1950s, when the first research paper on image texture appeared (Kaizer, 1955), image texture analysis has been another active research topic in computer vision and image processing. Texture effectively describes the properties of elements constituting the object surface, thus texture measurements are believed to contain substantial information for the pattern recognition of objects (Amadasun and King, 1989). Although texture can be roughly defined as the combination of some innate image properties, including fineness, coarseness, smoothness, granulation, randomness, lineation, hummocky, etc., a strictly scientific definition for texture has still not been determined (Haralick, 1979). Accordingly, there is no ideal method for measuring texture. Nevertheless, a great number of methods have been developed, and these are categorized into statistical, structural, transform-based, and model-based methods (Zheng et al., 2006a). These methods capture texture measurements in two different ways – by the variation of intensity across pixels, and by the intensity dependence between pixels and their neighboring pixels (Bharati et al., 2004).

5.1 Statistical methods In statistical methods, a matrix containing the higher order of image histograms is constructed from the intensities of pixels and their neighboring pixels. Statistics of matrix elements are then obtained as texture measurements. Statistical methods are effective in capturing micro-texture but are not ideal for analyzing macro-texture (Haralick, 1979), and thus they are suitable for analysis images from video cameras. Some of the applications include classification of beef tenderness (Li et al., 1999), identification of grains (Paliwal et al., 2003a, 2003b), and sorting of apples (Fernández et al., 2005). Currently developed statistical methods include the co-occurrence matrix (Haralick et al., 1973), the run-length matrix (Galloway, 1975), and the neighboring dependence matrix (Sun and Wee, 1983). 5.1.1 Co-occurrence matrix The co-occurrence matrix P is built according to the intensity co-occurrence between pixels and their neighboring pixels, which can be described by equation (3.25): max(|x 1 − x 2 |, |y − y |) = d 1 2 P(i, j, d, θ) = N (x 1 , y 1 ), (x 2 , y 2 ) ∈ W × W ((x 1 , y 1 ), (x 2 , y 2 )) = θ I(x 1 , y ) = i, I(x 2 , y ) = j 1 2 (3.25)


where i and j are two different intensity values; (x1 , y1 ) and (x2 , y2 ) indicate two pixels with the distance d and orientation θ; and W is the size of images. The matrix is normalized, and texture measurements consisting of fourteen statistics are obtained from it (Haralick et al., 1973). However, only seven of these are rated as important texture measurements (Gao and Tan, 1996a, 1996b; Zheng et al., 2006a), and these are listed in the appendix to this chapter. 5.1.2 Run-length matrix

Extraction of the run-length matrix R can be described by equation (3.26): L(pr) = i R(i, j, T ) = N pr I(pr) = j

(3.26)

where T is the threshold used for merging pixels into pixel-runs, r indicates pixel-runs, L is the length of pixel-runs, and I is the average intensity of pixel-runs. A pixel-run is a chain of connecting pixels with the similar intensity in the same row. Similar to the co-occurrence matrix, the run-length matrix is normalized and texture measurements are obtained with five statistical approaches (Galloway, 1975), which are also presented in the appendix. 5.1.3 Neighboring dependence matrix The neighboring dependence matrix (NDM) is dependent on two parameters, i.e. distance d and threshold T . Construction of the NDM is described by equation (3.27): I(x, y) = i (3.27) Q(i, j, d, T ) = N (x, y) N (x 1 , y ) |I(x, y) − I(x 1 , y 1 )| ≤ T = j 1 max(|x − x 1 |, | y − y |) ≤ d 1

where (x, y) and (x1 , y1 ) denote a pixel and its neighboring pixel. The NDM is normalized before the extraction of statistical measurements (see appendix) for texture description.

5.2 Structural methods Structural methods are based on some textural elements or structural primitives that occur repeatedly under the constraint of certain placement rules (Starovoitov et al., 1998). This is particularly popular in the analysis of textile (Palm, 2004). However, in the food industry, because the texture patterns in food images are very irregular, it is impossible to summarize a textural element or a structural primitive that can describe the texture constitution of food surfaces (Zheng et al., 2006a). Structural methods are therefore rarely used in the food industry and are not further discussed here.

5.3 Transform-based methods Transform-based methods extract texture measurements from images that are transformed from the original image using the convolution mask, Fourier transform, and

Texture 69

wavelet transform methods. Adjusted by parameters used during image transform, transform-based methods are suitable for both micro-texture and macro-texture patterns. However, the problem with transform-based methods is the greatly increased computation and storage load required while processing the transformed images, which will significantly reduce analysis speed. This is undesirable in the food industry, especially for on-line food quality inspection, because the inspecting process of every product needs to be accomplished within the time limit for conveying the product through the evaluation system. 5.3.1 Convolution mask With the convolution mask (CM), images are transformed by equation (3.28) from the spatial domain into the feature domain for the revelation of objects such as edges, spots, and lines (Patel et al., 1996). N(k, l)I(x + k, y + l) (3.28) I (x, y) = k

l

where I is the intensity of the transformed image from which texture measurements can be obtained by statistics, mostly mean and standard deviation. The most popular CM used to extract image texture is the Law’s mask, consisting of nine operators that are obtained by the multiplication of three vectors – [−1, 0, 1], [1, 2, 1], and [−1, 2, −1]. Another CM, the Gabor filter, has become more and more popular in texture classification in recent years, because the Gabor filter processes and extracts texture measurements with regard to three important parameters: space, frequency, and orientation. However, further detail of the Gabor filter is beyond our discussion here; interested readers might refer to the works by Daugman (1985), Kruizinga and Petkov (1999), and Setchell and Campbell (1999). 5.3.2 Fourier transform

Images are transformed into new forms by Fourier transform (FT) with regard to their spatial frequency of pixel intensities. From the FT magnitude images, texture measurements relating to the variation of pixel intensity can be obtained by statistical means. As images are in the form of two-dimensional matrices with discrete intensity values, a two-dimensional discrete FT is normally applied, which can be typically written as in equation (3.29): F (v x , v y ) =

y −1 N x −1 N

f (x, y) e−2j(2π/N x )v x x e−2j(2x/N y )v y y

(3.29)

x=0 y=0

where v denotes the Fourier coefficients. FT has been used in the food industry for measuring the color changes in the surface of chocolate (Briones and Aguilera, 2005). 5.3.3 Wavelet transform The use of wavelet transform (WT) to extract texture measurements is based on the multiresolution representation scheme, which is believed to be a formal representation


for any entities, including image texture (Mallat, 1989; Meyer, 1994). With WT, images are decomposed into different resolutions from which texture measurements regarding the different textural properties, from global texture at coarse resolution to local texture at fine resolution, can be obtained. Performance of WT has been found to exceed that of statistical methods in the food industry, including in the prediction of the chemical and physical properties of beef (Huang et al., 1997) and the sensory characteristics of pork (Cernadas et al., 2005). Three two-dimensional wavelets in three different directions – horizontal (along the x axis), vertical (along the y axis), and diagonal (along y = x), are first defined respectively as follows: H (x, y) = φ(x)ψ(y)

(3.30)

V (x, y) = ψ(x)φ(y)

(3.31)

D (x, y) = ψ(x)ψ(y)

(3.32)

where φ is the scaling function, and ψ is the one-dimensional wavelet. Afterwards, wavelet decomposition can be performed using equations (3.33)–(3.36), as proposed by Mallat (1989): Nx Ny A2i = I(x, y)φ2i (x − 2−i n)φ2i (y − 2−i m)dxdy (3.33) −N x

H2i =

−N x

V 2i =

Nx

−N x

D2i =

Nx

Nx

−N x

−N y

Ny

−N y

Ny

−N y

Ny

−N y

I(x, y)φ2i (x − 2−i n)ψ2i (y − 2−i m)dxdy

(3.34)

I(x, y)ψ2i (x − 2−i n)φ2i (y − 2−i m)dxdy

(3.35)

I(x, y)ψ2i (x − 2−i n)ψ2i (y − 2−i m)dxdy

(3.36)

where A, H , V , and D represent the approximation, horizontal signals, vertical signals, and diagonal signals, respectively, of the original image at the resolution of 2i . Parameters m and n stand for two sets of integers. An illustration of wavelet transform for beef images is displayed in Figure 3.6 (Zheng et al., 2006e).

5.4 Model-based methods In model-based methods, a model with unknown coefficients simulating the dependence of pixels and their neighboring pixels is first set up. By regressing the model with information from images, coefficients can be calculated as texture measurements. The different models developed have led to the different model-based methods, i.e. fractal models and the autoregressive model. 5.4.1 Fractal model

Surface intensity, showing the intensity value of pixels against their coordinates of an image, is obtained and assumed to be a fractal (Pentland, 1984), which is defined

Texture 71

Stage 4 Stage 3 Stage 2

(a)

Stage 1

(b)

Figure 3.6 Wavelet transform of a beef image (Zheng et al., 2006e): (a) original image; (b) wavelet transform of the region within the white boundary in (a).

as an object that remains the same regardless of the scale of observation (Quevedo et al., 2002). Texture measurements are thus obtained by the fractal dimension (FD), i.e. the dimension of the fractal (surface intensity in images), and can be determined by equation (3.37): L(φ) = Cφ1−FD

(3.37)

where L is a unit measurement such as perimeter, surface area, or volume; φ indicates the scale used; C is a constant associated with the unit measurement; and FD can be determined by a logarithmic regression against the observation scale φ. Employment of the different unit measurements will lead to the different fractal methods, such as the blanket method, the box counting method, and the frequency domain method (Quevedo et al., 2002). Fractal models are useful for describing the surface variation of food products such as pumpkin and chocolate (Quevedo et al., 2002). 5.4.2 Autoregressive model The autoregressive model, which is a stochastic model-based approach, explicitly describes the spatial relationship between pixels and their neighboring pixels while characterizing image texture (Kartikeyan and Sarkar, 1991). The dependency between pixels and their neighboring pixels in an image is expressed as a linear model, whose coefficients are later determined as texture measurements by regressing the model (Haralick, 1979; Thybo et al., 2004). However, there is no fast way to compute the regression coefficients, and thus the method is not commonly used in the food industry.


6 Combined measurements Recently, there has been a trend towards using more than one kind of object measurement (size, shape, color, and texture) in the applications of computer vision in the food industry. This is driven by two factors. The first is the rapid development of computer hardware, which has significantly increased the computing speed and computer storage, and therefore the number of considered object measurements has little or no impact on the computing speed. The second is based on the fact that quality evaluation is the most important issue that computer vision is used for in the food industry. Food quality is complex, being determined by the combination of sensory, nutritive, hygienic-toxicological, and technological properties (McDonald, 2001). More than one quality attribute will therefore be considered in most of the manual food quality grading systems. Furthermore, both geometrical measurements (size and shape) and surface measurement (color and texture) provide useful information regarding defect detection and the class discrimination of food products (Paliwal et al., 2003, 2003b; Diaz et al., 2004). It is therefore of great significance that the precision of computer vision systems can be improved when more object measurements are taken into account. For instance, the correlation coefficient has been found to be only 0.30 when using marbling characteristics (size measurements) and color measurements to indicate beef tenderness, whereas introducing texture measurements into the classification variables significantly increased the correlation coefficient, to 0.72 (Li et al., 1999).

7 Conclusions There are four kinds of object measurements that can be obtained from images – size, shape, color, and texture – and which contain significant information for food quality evaluation. Size and shape are two geometrical measurements, while color and texture are measurements of the object surface. Area, perimeter, width, and length are four of the primary measurements of object size. Area and perimeter are preferable to length and width, because they are more reliable and more easily extracted. Shape measurements can be categorized into two groups – size-dependent measurements (SDM) and size-independent measurements (SIM). The former work mostly for objects whose shape is more or less regular, while the latter are especially suitable for describing shapes with great irregularities. Color measurements are dependent on the color spaces used, which include hardware-orientated, human-orientated, and instrumental spaces. Hardware-orientated spaces are developed for the purpose of facilitating computer hardware processes; human-orientated spaces are aimed to help the human understanding of color; and instrumental spaces are employed for the comparison of computer measurements with those obtained from instruments. Techniques that are available for the extraction of texture measurements include statistical, structural, transform-based, and model-based methods. Statistical methods

Nomenclature 73

are competent for the analysis of micro-texture patterns. Although transform-based methods are suitable for both micro- and macro-texture patterns, a great deal of computation and computer storage is required. The model-based methods are limited by the lack of a fast way to regress the model. By the proper integration of different types of object measurements, the accuracy of computer vision for food quality inspection may be increased.

Nomenclature ε θ µ φ ψ A a a∗ Bˆ b∗ C D F f ˆ G H Hˆ I, I Iˆ i, j, k, l L Lˆ M m, n N P p, q pr Q ˆ Q R Rˆ r

uncorrelated sequence angel ensemble mean scaling function one-dimensional wavelet direction two-dimensional wavelet approximation coefficients color component of a∗ color component of blue color component b∗ constant diagonal signal Fourier transform function color component of green horizontal signal color component of hue intensity color component of intensity index parameters unit measurement color component of luminance moments set of integers number of elements in the set co-occurrence matrix order of the moments pixel-run neighboring dependence matrix color component of quadrature run-length matrix color component of red radius


Sˆ T ν u u , u u∗ V ν , ν ν∗ W Xˆ X , Y , Z x, y, x1 , y1 , x2 , y2 x¯ , y¯ Yˆ Zˆ

color component of saturation threshold Fourier coefficients zero mean stationary random sequence parameters used to calculate u∗ color component color component of u∗ vertical signal parameters used to calculate v∗ color component color component of v∗ size of images color component of X values of XYZ space at standard point D65 coordinates center of mass color component of Y color component of Z

Subscripts x, y H D V

coordinates horizontal signal diagonal signal vertical signal

Abbreviations: 1-D one-dimensional 2-D two-dimensional 3-D three-dimensional CM convolution mask FD fractal dimension FT Fourier transform SDM size-dependent measurements SIM size-independent measurements WT wavelet transform

Appendix Statistical measurements of co-occurrence matrix Angular second moment (ASM): ASM =

k

l

P 2 (k, l)

Appendix 75

Contrast (CT): ⎞

⎛ CT =

N j=0

⎟ ⎜ j2 ⎜ P(k, l)⎟ ⎠ ⎝ k l |k−l|=j

Mean value (µ): µ=

k

kP(k, l)

l

Sum of squares (SOS): SOS(σ 2 ) =

k

(k − µ)2 P(k, l)

l

Correlation (CR): k

CR =

(kl)P(k, l) − µ2

l

σ2

Inverse difference moment (IDM): IDM =

k

l

1 P(k, l) 1 + (k − l)2

Entropy (ET): ET = −

k

P(k, l) log (P(k,l))

l

Statistical measurements of run-length matrix Short run (SR): R(k, l) k

l

SR = k

l2 R(k,l)

l

Long run (LR): k

l 2 R(k, l)

l

LR = k

l

R(k, l)


Non-uniformity (NU): 2

NU =

k

R(k, l)

l

k

R(k, l)

l

Run-length non-uniformity (RLE): RLE =

l

2 R(k, l)

k

k

R(k, l)

l

Run percent (RP) describing the grainy of images: R(k, l) k

l

k

l

RP =

lR(k, l)

Statistical measurements of neighboring dependence matrix Small number emphasis (SNE): Q(k, l) k

l

k

l

SNE =

l2 Q(k, l)

Large number emphasis (LNE): k

l 2 Q(k, l)

l

LNE = k

Second moment (SM):

k

Q2 (k, l)

l

SE = k

Q(k, l)

l

l

Q(k, l)

References 77

Number of non-uniformity (NNU): l

SMT =

EM =

k

l

Q(k, l)

k

k

Entropy of the matrix (EM):

2

Q(k, l)

l

Q(k, l) log (Q(k, l))

k

Q(k, l)

l

References Amadasun M, King R (1989) Textural features corresponding to textural properties. IEEE Transactions on Systems, Man, and Cybernetics, 19, 1264–1274. Ballard DA, Brown CM (1982) Computer Vision. Englewood Cliffs: Prentice-Hall. Baxes GA (1994) Digital Image Processing Principle and Applications. New York: John Wiley & Sons. Bharati MH, Liu JJ, MacGregor JF (2004) Image texture analysis: methods and comparisons. Chemometrics and Intelligence Laboratory Systems, 72, 57–71. Briones V, Aguilera JM (2005) Image analysis of changes in surface color of chocolate. Food Research International, 38, 87–94. Cernadas E, Carrión P, Rodriguez PG, Muriel E, Antequera T (2005) Analyzing magnetic resonance images of Iberian pork loin to predict its sensorial characteristics. Computer Vision and Image Understanding, 98, 345–361. Collewet G, Bogner P, Allen P, Busk H, Dobrowolski A, Olsen E, Davenel A (2005) Determination of the lean meat percentage of pig carcasses using magnetic resonance imaging. Meat Science, 70, 563–572. Currie AJ, Ganeshanandam S, Noiton DA, Garrick D, Shelbourne CJA, Orgaguzie N (2000) Quantitative evaluation of apple fruit shape (Malus × domestica Borkh.) by principal component analysis of Fourier descriptors. Euphytica, 111, 221–227. Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of Optical Society of America, 2, 1160–1169. Devaux MF, Barakat A, Robert P, Bouchet B, Guillon F, Navez B, Lahaye M (2005) Mechanical breakdown and cell structure of mealy tomato pericarp tissue. Postharvest Biology and Technology, 37, 209–221. Diaz R, Gil L, Serrano C, Blasco M, Moltó E, Blasco J (2004) Comparison of three algorithms in the classification of table olives by means of computer vision. Journal of Food Engineering, 61, 101–107.


Du CJ, Sun D-W (2004a) Recent development in the applications of image processing techniques for food quality evaluation. Trends in Food Science & Technology, 15, 230–249. Du CJ, Sun D-W (2004b) Shape extraction and classification of pizza base using computer vision. Journal of Food Engineering, 64, 489–496. Du CJ, Sun D-W (2005) Comparison of three methods for classification of pizza topping using different color space transformations. Journal of Food Engineering, 66, 277–287. Du CJ, Sun D-W (2006) Learning techniques used in computer vision for food quality evaluation: a review. Journal of Food Engineering, 72, 39–55. Faucitano L, Huff P, Teuscher F, Gariepy C, Wegner J (2005) Application of computer image analysis to measure pork marbling characteristics. Meat Science, 69, 537–543. Fernández L, Castillero C, Aguilera JM (2005) An application of image analysis to dehydration of apple discs. Journal of Food Engineering, 67, 185–193. Galloway MM (1975) Texture analysis using grey level run lengths. Computer Vision, Graphics, and Image Processing, 4, 172–179. Gao X, Tan J (1996a) Analysis of expended-food texture by image processing part I: geometric properties. Journal of Food Process Engineering, 19, 425–444. Gao X, Tan J (1996b) Analysis of expended-food texture by image processing part II: mechanical properties. Journal of Food Process Engineering, 19, 445–456. Ghazanfari A, Irudayaraj J (1996) Classification of pistachio nuts using a string matching technique. Transactions of the ASAE, 39, 1197–1202. Hanbury A (2002) The taming of the hue, saturation, and brightness color space. In CVWW ’02 – Computer Vision Winter Workshop (Widenauer H, Kropatsch WG, eds). Autriche: Bad Aussee, pp. 234–243. Haralick RM (1979) Statistical and structural approaches to texture. Proceeding of the IEEE, 67, 786–804. Haralick RM, Shanmugan K, Dinstein I (1973) Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, 3, 610–621. Huang Y, Lacey RE, Moore LL, Miller RK, Whittaker AD, Ophir J (1997) Wavelet textural features from ultrasonic elastograms for meat quality prediction. Transactions of the ASAE, 40, 1741–1748. Jain AK (1989) Fundamentals of Digital Image Processing. Englewood Cliffs: Prentice-Hall. Kaizer H (1955) A quantification of texture on aerial photographs. Technology Note 121, AD 69484, Boston University Research Laboratory, Boston, MA, USA. Kartikeyan B, Sarkar A (1991) An identification approach for 2-D autoregressive models in describing textures. Graphical Models and Image Processing, 53, 121–131. Kashyap RL, Chellappa R (1981) Stochastic models for closed boundary analysis: representation and reconstruction. IEEE Transactions on Information Theory, 27, 627–637. Katsumata N, Matsuyama Y (2005) Database retrieval for similar images using ICA and PCA bases. Engineering Applications of Artificial Intelligence, 18, 705–717. Kruizinga P, Petkov N (1999) Nonlinear operator for oriented texture. IEEE Transactions on Image Processing, 8, 1395–1407.

References 79

Lana MM, Tijskens LMM, van Kooten O (2005) Effects of storage temperature and fruit ripening on firmness of fresh cut tomatoes. Postharvest Biology and Technology, 35, 87–95. Leemans V, Destain MF (2004) A real-time grading method of apple based on features extracted from defects. Journal of Food Engineering, 61, 83–89. Li J, Tan J, Martz FA, Heymann H (1999) Image texture features as indicators of beef tenderness. Meat Science, 53, 17–22. MacAdam DL (1970) Sources of Color Science. Cambridge: MIT Press. Mallat SG (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 674–693. McDonald K (2001) Effect of Vacuum Cooling on Processing Time, Mass Loss, Physical Structure and Quality of Large Cooked Beef Products. PhD Thesis, University College Dublin, National University of Ireland. Meyer Y (1994) Wavelets: Algorithms & Applications. Philadelphia: Society for Industrial and Applied Mathematics. Mulchrone KF, Choudhury KR (2004) Fitting an ellipse to an arbitrary shape: implication for strain analysis. Journal of Structural Geology, 26, 143–153. O’Sullivan MG, Byrne DV, Martens H, Gidskehaug LH, Andersen HJ, Martens M (2003) Evaluation of pork color: prediction of visual sensory quality of meat from instrumental and computer vision methods of color analysis. Meat Science, 65, 909–918. Paliwal J, Visen NS, Jayas DS, White NDG (2003a) Cereal grain and dockage identification using machine vision. Biosystems Engineering, 85, 51–57. Paliwal J, Visen NS, Jayas DS, White NDG (2003b) Comparison of a neural network and a non-parametric classifier for grain kernel identification. Biosystems Engineering, 85, 405–413. Palm C (2004) Color texture classification by integrative co-occurrence matrices. Pattern Recognition, 37, 965–976. Patel D, Davies ER, Hannah I (1996) The use of convolution operators for detecting contaminants in food images. Pattern Recognition, 29, 1019–1029. Pentland AP (1984) Fractal-based description of natural scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 661–674. Quevedo R, Carlos LG, Aguilera JM, Cadoche L (2002) Description of food surfaces and microstructural changes using fractal image texture analysis. Journal of Food Engineering, 53, 361–371. Rossel RAV, Minasny B, Roudier P, McBratney AB (2006) Color space models for soil science. Geodema, in press. Russ JC (1999) Image Processing Handbook, 3rd edn. Boca Raton: CRC Press. Schwarcz HP, Shane KC (1969) Measurement of particle shape by Fourier analysis. Sedimentology, 13, 213–231. Setchell CJ, Campbell NW (1999) Using color Gabor texture features for scene understanding. Proceeding of 7th International Conference on Image Processing and Its Application, pp. 372–376, Manchester, UK. Srikaeo K, Furst JE, Ashton JF, Hosken RW (2006) Microstructural changes of starch in cooked wheat grain as affected by cooking temperatures and times. LWT – Food Science and Technology, 39, 528–533.


Starovoitov VV, Jeong SY, Park RH (1998) Texture periodicity detection: features, properties, and comparisons. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 28, 839–849. Sun C, Wee WG (1983) Neighbouring grey level dependence matrix for texture classification. Computer Vision, Graphics, and Image Processing, 23, 341–352. Thybo AK, Szczypiński PM, Karlsson AH, Dønstrup S, Stødkilde-Jørgensen HS, Andersen HJ (2004) Prediction of sensory texture quality attributes of cooked potatoes by NMR-imaging (MRI) of raw potatoes in combination with different imaging analysis methods. Journal of Food Engineering, 61, 91–100. Tu K, Jancsók P, Nicolaï B, Baerdemaeker JD (2002) Use of laser-scatting imaging to study tomato-fruit quality in relation to acoustic and compression measurements. International Journal of Food Science and Technology, 35, 503–510. Wyszecki G, Stiles WS (1982) Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd edn. New York: John Wiley & Sons. Young T (1802) On the theory of light and colors. Philosophical Transactions of the Royal Society of London, 92, 20–71. Zheng C, Sun D-W, Zheng L (2006a) Recent development of image texture for evaluation of food qualities – a review. Trends in Food Science & Technology, 17, 113–128. Zheng C, Sun D-W, Zheng L (200b) Recent developments and applications of image features for food quality evaluation and inspection – a review. Trends in Food Science & Technology, 17, 642–655. Zheng C, Sun D-W, Zheng L (2006c) Estimating shrinkage of large cooked beef joints during air-blast cooling by computer vision. Journal of Food Engineering, 72, 56–62. Zheng C, Sun D-W, Zheng L (2006d) Predicting shrinkage of ellipsoid beef joints as affected by water immersion cooking using image analysis and neural network. Journal of Food Engineering, 79, 1243–1249. Zheng C, Sun D-W, Zheng L (2006e) Classification of tenderness of large cooked beef joints using wavelet and Gabor textural features. Transactions of the ASAE, 49, 1447–1454. Zion B, Shklyar A, Karplus I (1999) Sorting fish by computer vision. Computers and Electronics in Agriculture, 23, 175–197. Zion B, Shklyar A, Karplus I (2000) In-vivo fish sorting by computer vision. Aquaculture Engineering, 22, 165–179.

Object Classification Methods Cheng-Jin Du and Da-Wen Sun Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland

1 Introduction The classification technique is one of the essential features for food quality evaluation using computer vision, as the aim of computer vision is ultimately to replace the human visual decision-making process with automatic procedures. Backed by powerful classification systems, computer vision provides a mechanism in which the human thinking process is simulated artificially, and can help humans in making complicated judgments accurately, quickly, and very consistently over a long period (Abdullah et al., 2004). Using sample data, a classification system can generate an updated basis for improved classification of subsequent data from the same source, and express the new basis in intelligible symbolic form (Michie, 1991). Furthermore, it can learn meaningful or non-trivial relationships automatically in a set of training data, and produce a generalization of these relationships that can be used to interpret new, unseen test data (Mitchell et al., 1996). Generally, classification identifies objects by classifying them into one of the finite sets of classes, which involves comparing the measured features of a new object with those of a known object or other known criteria and determining whether the new object belongs to a particular category of objects. Figure 4.1 shows the general classification system configuration used in computer vision for food quality evaluation. Using imageprocessing techniques, the images of food products are quantitatively characterized by a set of features, such as size, shape, color, and texture. These features are objective data used to represent the food products, which can be used to form the training set. Once the training set has been obtained, the classification algorithm extracts the knowledge base necessary to make decisions on unknown cases. Based on the knowledge, intelligent decisions are made as outputs and fed back to the knowledge base at the same time, which generalizes the method that inspectors use to accomplish their tasks. The computationally hard part of classification is inducing a classifier – i.e., determining the optimal values of whatever parameters the classifier will use. Classifiers can give Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


4

82 Object Classification Methods

Training set

Classification algorithm

Unknown case

Knowledge base

Decisionmaking

Output

Classification system Figure 4.1

The general configuration of the classification system.

simple yes or no answers, and they can also give an estimate of the probability that an object belongs to each of the candidate classes. A wide variety of approaches has been taken towards this task in the food quality evaluation. Among the applications where classification techniques have been employed for building a knowledge base, artificial neural network (ANN) and statistical approaches are the two main methods. Fuzzy logic and the decision tree have also been used for classification. Besides the above classical classification approaches, the support vector machine (SVM) is a currently emerging classification technique and has been demonstrated to be feasible for performing such a task. All these approached have a common objective: to simulate a human decision-maker’s behavior, while having the advantage of consistency and, to a variable extent, explicitness. The fundamentals of these classification techniques as applied for food quality evaluation will be discussed in detail in the following sections.

2 Artificial neural network Initially inspired by the biological nervous system, ANN approaches combine the complexity of some of the statistical techniques with the objective of machines learning to imitate human intelligence, which is characterized by their self-learning capability. The key element of ANN is the novel structure of the information-processing system for modeling the functionality of a nervous system. Through a learning process, like humans, it can solve specific problems such as classification. ANNs have applicability to a number of types of food product classification, including cereal grains (Luo et al., 1999; Paliwal et al., 2001), fruits (Kavdir and Guyer, 2002; Li et al., 2002), fish (Storbeck and Daan, 2001), meat (Li et al., 2001; Chao et al., 2002), and vegetables (Nagata and Cao, 1998; Shahin et al., 2002).

2.1 Structure of neural network A neural network is a collection of interconnected nodes or processing elements (PEs), each of which is a key element of an ANN and is relatively simple in operation. The common structure of a PE is shown in Figure 4.2. Each input path is associated with a standardized signal using a transfer function (TF) and weighting. A PE has many inputs from several of the “upstream” PEs in the network. All inputs are summed to

Artificial neural network 83

Inputs

Weights

Output

x0 x1

w0 w1

x2

w2

wn

xn Figure 4.2 Common structure of a processing element (+ = sum, TF = transfer function).

produce a non-linear function of its input. The PE then generates an output, and sends it “downstream” to the input paths of another group of PEs. The input weighting can be changed adaptively, which makes this PE very flexible and powerful. The algorithms for adjustment of weighting will be discussed in the following section. The transfer functions can be classified into three categories: linear, threshold, and sigmoid. The output of a linear function is proportional to the total weighted output. For the threshold function, the output is set at one of two levels, depending on whether the total input is greater than or less than some threshold value. Since sigmoid functions can obtain the output varying continuously but not linearly as the input changes, they are the most widely used transfer functions. Figure 4.3 illustrates the general topology of an ANN. The complete network represents a very complex set of interdependencies, and may incorporate any degree of non-linearity in theory. For food quality evaluation, very general functions can be modeled to transform physical properties into quality factors. ANN technology allows the extension of computer vision technology into the areas of color, content, shape, and texture inspection at near-human levels of performance, and can provide the decisionmaking and classification capabilities to succeed in these inspection tasks (Domenico and Gary, 1994). The input layer represents the raw information fed into the network, which normally consists of the image attributes of food products, such as size, shape, color, and texture. The input values are generally normalized, usually in the range of [0–1]. The number of PEs in an input layer is typically defined based on different attribute types and attribute domain. A neural network can have one or more hidden layers. Hidden layer(s) are constructed for the process of learning by computations on their node and arc weights. The activity of hidden layers is determined by the activities of the input PEs and the weighting on the connections between the input and the hidden PEs. The result of classification is the output of a PE in the output layer. Typically, there is one output PE for each class. The behavior of the output layer depends on the activity of the hidden


Input layer

Output layer

…

…

…

…

…

…

Figure 4.3

Hidden layer (there may be several hidden layers)

The general topology of an artificial neural network.

layers, the weights and transfer functions between the hidden and output layers. The PEs of input, hidden, and output layers are connected by arcs. Each arc is assigned an initial random weighting, usually [−0.5 . . . 0.5], used in training, and may be modified in the learning process. The number of layers and the number of PEs per layer are the “art” of an ANN designer. There is no quantifiable, best answer to the structure of an ANN for food classification. Generally, as the complexity in the relationship between the input data and the desired output increases, the number of PEs in the hidden layer should also increase. The single-layer organization constitutes the most general case, and is of more potential computational power than hierarchically structured multi-layer organizations. The additional hidden layer(s) might be required when the process being modeled is separable into multiple stages. The number of PEs in the hidden layer(s) should be less than the amount of training data available. If too many PEs are used, the training set will be memorized and lead to over-fitting. As a result, generalization of the data will not occur, and the network will become useless on new data sets. However, too few PEs will reduce the classification accuracy. The exact number of PEs in the hidden layer(s) should be determined via experimentation.

2.2 Learning process The knowledge of the ANN is contained in the values of connection weights. Learning involves adjustments to the values of weighting by passing the information about response success backward through the network. Modifying the knowledge stored in an ANN as a function of experience implies a learning rule of how to adapt the values of the weights. For a simple PE, the fixed incremental rule could be used to adjust weighting. The algorithm could be described as follows: 1. Initializing weights with small random numbers 2. Selecting a suitable value for the learning rate coefficient γ, ranging from 0 to 1

Artificial neural network 85

3. Running a sample feature vector x = (x1 , x2 , . . . , xd ) with d-dimension from a training set as input 4. Applying the summation of weighted input S = di=0 wi xi and transfer function tf to obtain an output y = tf (S) 5. Comparing the output with the expected class c from the training set; if the output does not match, modifying arc weights according to wi = wi + γ(c − y)xi 6. Running the next sample and repeat steps 3–5 7. Repeating steps 3–6 until the weights converge. The concept of this algorithm is to find a linear discriminant plane, by moving a fixed distance, where no misclassification error occurs. If the feature vectors are linearly separable, the algorithm will converge and a correct, error-free solution is found. Unfortunately, most feature vectors of food products are non-linearly separable. To cope with this problem, one of the alternative algorithms developed for adjusting the values of weights is the delta rule, which is used in feed-forward networks. The weights are changed in proportion to the error δ in the equation (4.1): w i (k + 1) = w i (k) + γδxi (k) = w i (k) + γ[c(k) − S(k)]x i (k)

(4.1)

where k indicates the kth iteration of the classifier, and c(k) is the class of the kth training pattern. Another solution is the back-propagation learning rule proposed by Rumelhart et al. (1986), which has become one of the most important methods for training neural networks. In order to avoid confusion, a clear notation is described first: [s]

yj

output state of jth PE in layer s

[s] wji S[s] j

connection weight joining ith PE in layer (s − 1) to jth PE in layer s summation of weighted inputs to jth PE in layer s

A PE in the output layer determines its activity by two steps. First, it computes [o] the total weighted input Sj using the formula: [o]

Sj =

[o] [o−1]

w ji y i

(4.2)

i [o−1]

is the output state of the ith unit in the previous layer. Then the PE where yi [o] [o] calculates the output state yj using transfer function of the total weighted input Sj . Typically, the following sigmoid function is used: 1 [o] [o] yj = tf Sj (4.3) = [o] −S 1+e j Once the activities of all the output units have been determined, the network computes the global error function E, which is given by 1 [o] 2 (4.4) cj − yj E= 2 j


[o]

where cj denotes the desired output, and yj denotes the actual output produced by the network with its current set of weights. Based on equations (4.2)–(4.4) described above, a standard back-propagation algorithm is given as follows: 1. Initializing weights with small random numbers 2. Selecting a suitable value for the learning rate coefficient γ, ranging from 0 to 1 3. Running a sample feature vector x from the training set as input, and obtaining an output vector y[o] at the output layer of the network 4. Calculating the local error and delta weight for each PE in the output layer as follows: [o] ej

=−

[o]

∂E [o]

∂Sj

=−

∂E ∂y j [o]

[o]

∂y j ∂Sj

[o] [o] = c j − y j tf Sj

(4.5)

[o] [o] [o] = yj 1 − yj , if the sigmoid function is used as the transfer where tf Sj function. The delta weight of an output layer node can be given by: [o]

[o]

w ji = −γej

(4.6)

5. Calculating the local error and delta weight for each PE in the hidden layers using the following equations respectively: [s] [s] [s+1] [s+1] ej = tf Sj w ij ei (4.7) i [s]

[s] [s−1]

w ji = −γej y i

(4.8)

6. Updating all the weights in the network by adding the delta weights to the corresponding previous weights 7. Running the next sample and repeating steps 3–6 8. Repeating steps 3–7 until the changes in weights are reduced to some predetermined level.

3 Statistical classification Statistical classification (SC) utilizes the statistical properties of the observations from the training set. It is generally characterized by having an explicit underlying probability model, for example Bayesian theory, which is mathematically rigorous and provides a probabilistic approach to inference. Based on a well-established field of mathematics, SC has been proven successful in applications of computer vision for quality evaluation of food products. Generally, there are three kinds of SC techniques used in applications: Bayesian classification, discriminant analysis, and nearest neighbor.

Statistical classification 87

3.1 Bayesian classification Bayesian classification is a probabilistic approach to learning and inference based on a different view of what it means to learn from data, in which probability is used to represent uncertainty about the relationship being learnt. Before we have seen any data, our prior opinions about what the true relationship might be are expressed in a probability distribution. After we look at the data, our revised opinions are captured by a posterior distribution. Bayesian learning can produce the probability distributions of the quantities of interest, and make the optimal decisions by reasoning about these probabilities together with observed data (Mitchell, 1997). In order to improve the objectivity of the inspection, Bayesian classifiers have been implemented for the automated grading of apples (Shahin et al., 1999), mandarins and lemons (Aleixos et al., 2002), raisins (Okamura et al., 1993), carrots (Howarth and Searcy, 1992), and sweet onions (Shahin et al., 2002). Suppose there are n classes (c1 , c2 , . . . , cn ) and A summarizes all prior assumptions and experience, the Bayesian rule tells how the learning system should update its knowledge as it receives a new observation. Before giving a new observation with feature vector x, the learning system knows only A. Afterwards, it knows xA, i.e. x and A. Bayes’ rule then tells how the learning system should adapt P(ci |A) into P(ci |xA) in response to the observation x as follows: P(c i |xA) =

P(c i |A)P(x|c i A) P(x|A)

(4.9)

where P(ci |xA) is usually called the posterior probability and P(ci |A) the prior probability of class ci (it should be noted that this distinction is relative to the observation; the posterior probability for one observation is the prior probability for the next observation); P(x|ci A) is the class-conditional probability density for observation x in class ci and the prior assumptions and experience A. Both P(ci |A) and P(x|ci A) could be determined if (c1 , c2 , . . . , cn ) are exhaustive and mutually exclusive – in other words if exactly one of ci is true while the rest are false. P(x|A) is the conditional probability of the prior assumptions and experience Z, and can be derived by P(x|A) = P(c k |A)P(x|c k A) (4.10) k

The Bayesian decision rule selects the category with minimum conditional risk. In the case of minimum-error rate classification, the rule will select the category with the maximum posterior probability. The classification procedure is then to compare the values of all the P(ci |xA) and assign the new observation to class ci if P(c i |xA) > P(c j |xA)

for all i = j

(4.11)

Figure 4.4 illustrates the structure of a Bayesian classifier. So far, we have explicitly denoted that the probabilities are conditional to the prior assumptions and experience A. In most cases the context will make it clear which are the prior assumptions, and usually A is left out. This means that probability statements like P(x) and P(ci |x) should be


P(c1|xA)

P(c2|xA) x

x∈c Max P(ci|xA)

P(cn|xA)

Figure 4.4

Structure of a Bayesian classifier.

understood to mean P(x|A) and P(ci |xA) respectively, where A denotes the assumptions appropriate for the context.

3.2 Discriminant analysis Discriminant analysis is a very useful multivariate statistical technique which takes into account the different variables of an object and works by finding the so called discriminant functions in such a way that the differences between the predefined groups are maximized. The obtained discriminant rules provide a way to classify each new object into one of the previous defined groups. Discriminant analysis has been demonstrated as plausible for the classification of apples (Leemans and Destain, 2004), corn (Zayas et al., 1990), edible beans (Chtioui et al., 1999), poultry carcasses (Park et al., 2002), mushrooms (Vízhányó and Felföldi, 2000) and muffins (Abdullah et al., 2000), and for individual kernels of CWRS wheat, CWAD wheat, barley, oats, and rye, based on morphological features (Majumdar and Jayas, 2000a), color features (Majumdar and Jayas, 2000b), and textural features (Majumdar and Jayas, 2000c). The most famous approach of discriminant analysis was introduced by Fisher for two class problems (Fisher, 1936). By considering two classes of d-dimensional observations x with means µ1 and µ2 , Fisher discriminant analysis seeks a linear combination of features w · x that has a maximal ratio of between-class variance to within-class variance as follows: w T MB w (4.12) w T MW w where MB = (µ1 − µ2 )(µ1 − µ2 )T and MW = i=1,2 dk=1 (xki − µi )(xki − µi )T are the between- and within-class scatter matrices respectively. The intuition behind maximizing J (w) is to seek a linear direction for which the projected classes are well separated. If the within-class scatter matrix MW has full rank, the maximum separation J(w) =

Statistical classification 89

occurs when w = M−1 w (µ1 − µ2 ). When MW is singular, it cannot be inverted. The problem can be tackled in different ways; one method is to use a pseudo inverse instead of the usual matrix inverse (Rao and Mitra, 1971). Fisher discriminant analysis is a very reasonable measurement of class separability. Several approaches could be applied to generalize it for more than two classes, for example the method developed by Rao (1948). The most common approach is to substitute variance for covariance and simple ratios for ratios of determinants, which is based on the fact that the determinant of a covariance matrix, known as generalized variance, is the product of the variances along principal component directions. Given a set of l d-dimensional samples represented by x, where each case belongs to one of n known classes, X is the l × d matrix of all the group of samples and U is its means, M is the n × d matrix of class means, and G is the l × n matrix of class membership matrix that indicates which class each sample belongs to (gij = 1 if and only if sample i is assigned to class j, or else gij = 0), then the within-class and between-class sample covariance matrices are: CMW =

(X − GM)T (X − GM) l−n

(4.13)

CMB =

(GM − U)T (GM − U) n−1

(4.14)

Then the problem of multiple discriminant analysis could be considered finding a d × (n − 1) projection matrix W for which the projected samples XW are well separated. Thus the two-class criterion consists of seeking the projection that maximizes the ratio of the determinants of the within-class to the between-class covariance matrices, and could be generalized as: J(W ) =

W T CMB W W T CMW W

(4.15)

The projection matrix W can be computed by solving the following generalized eigenvector problem: CMB W i = λi CMW W i

(4.16)

If the classes are Gaussian with equal covariance and their mean vectors are well separated, the discriminant can achieve the optimal result with the minimum classification error. However, when the distributions are non-Gaussian or the mean vectors of the two classes are close to each other, the performance of discriminant will be poorer.

3.3 Nearest neighbor As well as the Bayesian classification and discriminant analysis, the nearest-neighbor method is also feasible for classification of foods. For example, it has been applied to classify healthy and six types of damaged Canadian Western Red Spring wheat kernels using selected morphological and color features extracted from the grain sample


images (Luo et al., 1999). Nearest neighbor is a non-parametric classification technique performed by assigning the unknown case to the class most frequently represented among the nearest samples. Without a priori assumptions about the distributions from which the training examples are drawn, the nearest-neighbor classifier could achieve consistently high performance in spite of its simplicity. It involves a training set of both positive and negative cases. A new sample is classified by calculating the distance to the nearest training case; the sign of that point then determines the classification of the sample. The k-nearest-neighbor (k-NN) classifier extends this idea by taking the k nearest points, i.e. the closest neighbors around the new observation with feature vector x. The classification is usually performed by a majority voting rule, which states that the new sample to be assigned should be the label occurring most among the neighbors. Several design choices arise when using this classifier. The first choice is to find a suitable distance measurement; the second is the number of neighbors of k – choosing a large k generally results in a linear classifier, whereas a small k results in a non-linear one, which influences the generalization capability of the k-NN classifier. Furthermore, the design of the set of prototypes is also an important issue. The most common distance metric used to calculate the distances between samples is Euclidean distance. Given two samples xi and xj , the Euclidean distance between the two samples is defined as:

DE (x i , x j ) = x i − x j

(4.17)

Other measures can also be used, such as the city-block distance and Mahalanobis distance, defined respectively as follows: DC (x i , x j ) = |x ik − x jk | (4.18) k=1

DM (x i , x j ) = (x i − x j ) CM−1 (x i − x j )

(4.19)

where CM represents the covariance matrix. The city-block distance is also known as the Manhattan distance, boxcar distance or absolute value distance. It represents the distance between points in a city road grid, and examines the absolute differences between the coordinates of a pair of feature vectors. Mahalanobis distance takes the distribution of the points (correlations) into account, and is a very useful way of determining the “similarity” of a set of values from an “unknown” sample to a set of values measured from a collection of “known” samples. The Mahalanobis distance is the same as the Euclidean distance if the covariance matrix is the identity matrix. Choosing the correct k is a hard problem. Too large (or too small) a k may result in non-generalizing classifiers. The choice of k is often performed through the leaveone-out cross-validation method on the training set. Leave-one-out cross-validation (Martens and Martens, 2001) can make good use of the available data and provide an almost unbiased estimate of the generalization ability of a model. At the start, the first observation is held out as a single-element test set, with all other observations

Fuzzy logic 91

as the training set. After that, the second observation is held out, then the third, and so on. This of course still requires independent test sets for accurate error estimation and comparison of different k-NN classifiers. The design of the set of prototypes is the most difficult and challenging task. The simplest approach is to select the whole training set as prototypes. However, this simple approach requires huge memory and execution in large databases, and hence the size of prototypes should be reduced in practice. The strategies for reducing the number of stored prototypes can be divided into three types: condensing, editing, and clustering algorithms. Condensing algorithms aim to keep those points that are near the class border from the training data, which form the class boundaries (Hart, 1968). Editing algorithms retain those training data that fall inside the class borders, and tend to form homogeneous clusters since only the points that are at the centre of natural groups in the data are retained (Wilson, 1972). It is also feasible to use any clustering algorithm, such as k-means, to form a set of labeled prototypes (Devroye et al., 1996). The advantage for clustering algorithms is that prototypes are not constrained to training points, and thus more flexible classifiers can be designed.

4 Fuzzy logic Fuzzy logic is introduced as a representation scheme and calculus for uncertain or vague notions, and could provide a completely different method for applications such as the classification of food products. Compared with traditional classification techniques, fuzzy classification groups individual samples into classes that do not have sharply defined boundaries. It embodies the nature of the human mind in some sense, as the concepts of possibility and probability are emphasized in this logic. In contrast with the absolute values and categories in the traditional Boolean logic, it mimics more human behavior for decision-making and reasoning by extending the handling of the intermediate categories to partially true or partially false. Thus it can simulate the human experience of generating complex decisions using approximate and uncertain information. The application of fuzzy logic in food quality evaluation includes the grading of apples (Shahin et al., 2001) and tomatoes (Jahns et al., 2001). The introduction of fuzzy set theory by Zadeh (1965) marked the beginning of a new way of solving classification problems by providing a basis for a qualitative approach to the analysis of a complex system. By incorporating the basics of fuzzy set theory, in which linguistic or “fuzzy” terms rather than relationships between precise numerical values are employed to describe system behavior and performance, a classification system can make a decision in a similar way to humans. The fuzzy classifier is inherently robust, does not require precise inputs, and can obtain a definite conclusion even based upon vague, ambiguous, imprecise, and noisy input or knowledge. Figure 4.5 shows a typical structure of a fuzzy classification system, which essentially defines a non-linear mapping of the input data vector into a scalar output using fuzzy rules. If considering an input vector x, the first step for a fuzzy classification system is to transform crisp input variables into linguistic variables by creating fuzzy sets and membership functions. The second step is to construct a fuzzy rule base. By computing


Input x

Creating fuzzy sets and membership functions

Constructing fuzzy rule base

Producing fuzzy outputs

Defuzzification

Output y Figure 4.5

Structure of a fuzzy classification system.

the logical product for each of the effective rules, a set of fuzzy outputs is produced. Finally, the fuzzy outputs are processed and combined in some manner to produce a crisp (defuzzified) output.

4.1 Creating fuzzy sets and membership functions 4.1.1 Fuzzy set

The very basic notion of a fuzzy classification system is a fuzzy set. A fuzzy set S in a fuzzy space X could be represented as a set of ordered pairs:

S = {(x, τ(x)|x ∈ X )}

(4.20)

where x is a generic element, and τ(x) characterizes its grade of membership. In Boolean logic, every element is true or false – i.e. restricted to just two values, 1 or 0 – and thus imposes rigid membership. In contrast, fuzzy sets have more flexible membership requirements that allow for partial membership in a set. Each element of a fuzzy set has a degree of membership, which can be a full member (100 percent membership) or a partial member (between 0 and 100 percent membership) – i.e. the membership value assigned to an element can be 0, 1, or any value in between. Compared with the crisp sets in Boolean logic, fuzzy sets are more flexible in applications. The flexibility of fuzzy set design allows different relationships between the neighbor sets. Fuzzy sets in a fuzzy universe can be fully separated, or they can be arranged in an overlapping manner. Hence, in fuzzy logic the freedom of both shape and association of the fuzzy sets provides a broad base for applying fuzzy logic.

Fuzzy logic 93

The design of a series of fuzzy sets depends on the characteristics and complexity of the classification problem. Although some formal procedures have been proposed for obtaining fuzzy set mapping, there is still no theoretically universal method (Dutta, 1993). A principle called “minimum normal form,” which requires at least one element of the fuzzy set domain to have a membership value of one, is most widely used. 4.1.2 Membership function The mathematical function that defines the degree of an element’s membership in a fuzzy set is called the membership function. In literature, a variety of membership functions have been used, including linear, sigmoid, beta curve, triangular curve, and trapezoidal curve (Sonka et al., 1999). The more complex the membership functions are, the greater the computing overhead implement. The membership function is a graphical representation of the magnitude of participation of each input variable. The number 1 assigned to an element means that the element is in the set s, and 0 means that the element is definitely not in the set S. All other values mean a graduated membership of the set S. In such a way, the membership function associates a weight with each of the inputs that are processed, defines the functional overlap between inputs, and ultimately determines an output response. These weighting factors determine the degree of influence or of membership.

4.2 Constructing a fuzzy rule base A fuzzy rule base contains a set of fuzzy rules, whose forms are usually expressed in IF–THEN. Each fuzzy rule consists of two parts, i.e. an antecedent block (between the IF and THEN) and a consequent block (following THEN). Depending on the classification system, it may not be necessary to evaluate every possible input combination, since some may rarely or never occur. By making this type of evaluation, it can simplify the processing logic and perhaps even improving the fuzzy logic system performance. In fuzzy logic, the AND, OR, and NOT operators of Boolean logic are usually defined as the minimum, maximum, and complement, as Zadeh’s (1965) paper. So for the fuzzy variables x1 and x2 : NOT x 1 = (1 − truth(x 1 ))

(4.21)

x 1 AND x 2 = minimum(truth(x 1 ), truth(x 2 ))

(4.22)

x 1 OR x 2 = maximum(truth(x 1 ), truth(x 2 ))

(4.23)

There are also other operators, called linguistic hedges. Hedges play the same role as in fuzzy production rules that adjectives and adverbs play in English sentences, such as “very” or “somewhat.” By modifying the fuzzy set’s membership function, hedges allow the generation of fuzzy statements through a mathematical formula. According to their impact on the membership function, the hedges are divided into three groups: concentrator, dilator, and contrast hedges. The concentrator hedge intensifies the fuzzy region as τcon(S ) (x) = τSn (x), where n ≥ 1. In contrast, the dilator hedge dilutes the force 1/n of fuzzy set membership function by τdil(S ) (x) = τS (x). The contrast hedge changes


the nature of the fuzzy region by making it either less fuzzy (intensification) or more fuzzy (diffusion): 1 1/2 if τ is ≥ 0.5, τ(S ) = τ (S ) 2 S if τ < 0.5, τ(S ) = 1 −

1 1/2 τ (S ) 2 S

(4.24)

4.3 Producing fuzzy outputs and defuzzification The interpretation of an IF–THEN rule can be evaluated as follows. All fuzzy statements in the antecedent block are first mapped to a degree of membership between 0 and 1. If there are multiple parts in the antecedent, fuzzy logic operators are applied to resolve the antecedent to a single number between 0 and 1. After that, the conclusions of the consequent block are combined to form a logical sum. The fuzzy outputs for all rules are finally aggregated into a single composite output fuzzy set. The fuzzy set is then passed on to the defuzzification process for crisp output generation – that is, to choose one representative value as the final output. This process is often complex, since the resulting fuzzy set might not translate directly into a crisp value. Several heuristic defuzzification methods exist. One of them is the centroid method, which is widely used in the literature. This method finds the “balance” point of the solution fuzzy region by calculating the weighted mean of the output fuzzy region. The weighted strengths of each output member function are multiplied by their respective output membership function center points and summed. This area is then divided by the sum of the weighted member function strengths, and the result is taken as the crisp output. Besides the centroid method, the max method chooses the element with the highest magnitudes. This method produce a continuous output function and is easy to implement; however, it does not combine the effects of all applicable rules. The weighted averaging method is another approach that works by weighting each membership function in the output by its respective maximum membership value. Nonetheless, it fails to give increased weighting to more rule votes per output member function.

5 Decision tree The decision tree acquires knowledge in the form of a tree, which can also be rewritten as a set of discrete rules to make it easier to understand. The main advantage of the decision tree classifier is its ability to using different feature subsets and decision rules at different stages of classification. As shown in Figure 4.6, a general decision tree consists of one root node, a number of internal and leaf nodes, and branches. Leaf nodes indicate the class to be assigned to a sample. Each internal node of a tree corresponds to a feature, and branches represent conjunctions of features that lead to those classifications. For food quality evaluation using computer vision, the decision

Decision tree 95

Figure 4.6 A general decision tree structure; respectively.

, , and represent root, internal, and leaf nodes

tree has been applied to the problem of meat quality grading (Song et al., 2002) and the classification of “in the shell” pistachio nuts (Ghazanfari et al., 1998). The performance of a decision tree classifier depends on how well the tree is constructed from the training data. A decision tree normally starts from a root node, and proceeds to split the source set into subsets, based on a feature value, to generate subtrees. This process is repeated on each derived subset in a recursive manner until leaf nodes are created. The problem of constructing a truly optimal decision tree seems not to be easy. As one of the well-known decision tree methods, C4.5 is an inductive algorithm developed by Quinlan (1993); this is described in detail below. To build a decision tree from training data, C4.5 employs an approach which uses information theoretically measured based on “gain” and “gain ratio.” Given a training set TS, each sample has the same structure. Usually, the training set TS of food products is partitioned into two classes – AL (acceptable level) and UL (unacceptable level). The information (I) needed to identify the class of an element of TS is then given by |AL| |UL| |AL| |UL| I(TS) = − log2 − log2 (4.25) |TS| |TS| |TS| |TS| If the training set TS is partitioned on the basis of the value of a feature xk into sets TS1 , TS2 , . . . , TSn , the information needed to identify the class of an element of TS can be calculated by the weighted average of I (TSi ) as follows: I(x k , TS) =

n |TS i | i=1

|TS|

I(TS i )

(4.26)


The information gained on a given feature is the difference between the information needed to identify an element of TS and the information needed to identify an element of TS after the value of the feature has been obtained. Therefore, the information gained on xk is gain(x k , TS) = I(TS) − I(x k , TS)

(4.27)

The root of the decision tree is the attribute with the greatest gain. The process of building the decision tree is repeated, where each node locates the feature with the greatest gain among the attributes not yet considered in the path from the root. The gain measurement has disadvantageous effects regarding the features with a large number of values. To cope with this problem, the gain ratio is introduced instead of the gain. For example, the gain ratio of xk is defined as: gainratio(x k , TS) =

split(x k , TS) =

gain(x k ,TS) split(x k , TS)

n |TS i | i=1

|TS|

log2

|TS i | |TS|

(4.28) (4.29)

where split(xk , TS) is the information due to the split of TS on the basis of the value of feature xk . Sometimes, the decision tree obtained by recursively partitioning a training set as described above may become quite complex, with long and uneven paths. To deal with this shortcoming, the decision tree is pruned by replacing a whole sub-tree with a leaf node through an error-based strategy (Quinlan, 1993).

6 Support vector machine The support vector machine (SVM) is a state-of-the-art classification algorithm which has a good theoretical foundation in statistical learning theory (Vapnik, 1995). Instead of minimization of the misclassification on the training set, SVM fixes the decision function based on structural risk minimization to avoid the overfitting problem. It performs classification by finding maximal margin hyperplanes in terms of a subset of the input data between different classes. The subset of vectors defining the hyperplanes is called a support vector. If the input data are not linearly separable, SVM first maps the data into a high- (possibly infinite) dimensional feature space, and then classifies the data by the maximal margin hyperplanes. Furthermore, SVM is capable of classification in high-dimensional feature space with fewer training data. SVM was originally developed for the problem of binary classification. Recently, it has also been shown a great deal of potential in multi-class problems. As one of the relatively novel learning techniques, SVM has been successfully applied to some classification problems, such as electronic nose data (Pardo and Sberveglieri, 2002; Trihaas and Bothe, 2002) and bakery process data (Rousu et al., 2003), and pizza grading (Du and Sun, 2004, 2005a, 2005b).

Support vector machine 97

6.1 Binary classification The classification of food products into acceptable and unacceptable quality levels can be examined as a binary categorization problem. Suppose that there are l samples in the training data, and each sample is denoted by a vector xi , binary classification can be described as the task of finding a classification decision function f :xi → yi , yi ∈ {−1, +1} using training data with an unknown probability distribution P(x, y). Subsequently, the classification decision function f is used to correctly classify the unseen test data. If f (x) > 0, the input vector x is assigned to the class y = +1, i.e. the acceptable quality level, or to the class y = −1, i.e. the unacceptable quality level. The classification decision function f is found by minimizing the expected classification risk as follows: 1 CR(f ) = | y − f (x)|dP(x, y) (4.30) 2 Unfortunately, the expected classification risk shown in equation (4.30) cannot be calculated directly because the probability distribution P(x, y) is unknown. Instead, the “empirical risk” ERemp ( f ) is applied to approximate the expected classification risk on the training set (Burges, 1998): ERemp (f ) =

l 1 | y − f (x i )| 2l i=1 i

(4.31)

Although there is no probability distribution appearing in equation (4.31), the classification decision function f still cannot be found correctly because the empirical risk might differ greatly from the expected classification risk for small sample sizes. Structural risk minimization (SRM) is a technique suggested by Vapnik (1995) to solve the problem of capacity control in learning from “small” training data. With a probability of 1 − η (where 0 ≤ η ≤ 1), the following bound holds on the expected classification risk (Vapnik, 1995): VCD(log(2l/VCD)) − log(η/4) CR(f ) ≤ ERemp (f ) + (4.32) l where VCD is the Vapnik Chervonenkis dimension of the set of functions from which the classification decision function f is chosen. The second term on the right-hand side of equation (4.32) is the so-called “VC confidence.” SRM attempts to find the function for minimizing the upper bound by training. For the linearly separable training vectors xi , the classification function has the following form: f (x) = sgn(ωT x + b)

(4.33)

where ω is normal to the hyperplane and b is a bias term, which should satisfy the following conditions: y i (ωT x i + b) ≥ 1, i = 1, 2, . . . , l

(4.34)


SVM intends to find the optimal separating hyperplane that maximizes the margin between positive and negative samples. The margin is 2/ω, thus the optimal separating hyperplane is the one minimizing 12 ωT ω, subject to constraints shown in equation (4.34), which is a convex quadratic programming problem. For the linearly non-separable case, the constraints in equation (4.34) are relaxed by introducing a new set of non-negative slack variables {ξi |i = 1, 2, . . . , l} as the measurement of violation of the constraints (Vapnik, 1995), as follows: y i (ωT x i + b) ≥ 1 − ξi , i = 1, 2, . . . , l

(4.35)

The optimal hyperplane is the one that minimizes the following formula: l 1 T – ω ω+ λ ξi 2 i=1

(4.36)

where –λ is a parameter used to penalize variables ξi , subject to constraints in equation (4.35). For a non-linearly separable case, the training vectors xi can be mapped into a high dimensional feature space (HDFS) by a non-linear transformation ϕ(·). The training vectors become linearly separable in the feature space HDFS and then separated by the optimal hyperplane as described before. In many cases the dimension of HDFS is infinite, which makes it difficult to work with ϕ(·) explicitly. Since the training algorithm only involves inner products in HDFS, a kernel function k(xi , xj ) is used to solve the problem, which defines the inner product in HDFS: k(x i , x j ) = ϕ(x i ), ϕ(x j )

(4.37)

Besides a linear kernel, polynomial kernels and Gaussian radial basis function (RBF) kernels are usually applied in practice, which are defined as: k(x i , x j ) = (x i x j + b)m

(4.38)

k(x i , x j ) = exp(−x i − x j 2/2σ 2 )

(4.39)

where b is the bias term and m is the degree of polynomial kernels. The classification function then has the following form in terms of kernels:

l y i αi k(xi , x) + b (4.40) f (x) = sgn i=1

where αi can be obtained by solving a convex quadratic programming problem subject to linear constraints. The support vectors are those xi with αi > 0 in equation (4.40). To illustrate the performance of SVM classifiers, a two-dimensional data set with five samples for each class is shown in Figure 4.7, where the samples of class +1 are represented by the lighter dots and the samples of class −1 by the darker dots. The performance of a linear SVM is illustrated in Figure 4.8a. If the input data are not linearly separable, SVM first maps the data into a high-dimensional feature space using a kernel function, such as the polynomial kernel (equation (4.38)) and Gaussian


0.75 0.5 0.25 0.0 −0.2 −0.5 −0.7 −1.0 −1.0

−0.7

−0.5

−0.2 Class ⫹1

0.0

0.25

0.5

0.75

Class ⫺1

Figure 4.7 An illustrated data set.

RBF kernel (equation (4.39)), and then classifies the data by the maximal margin hyperplanes as shown in Figures 4.8a and 4.8b, respectively.

6.2 Multi-classification Although SVM was originally developed for the problem of binary classification, several SVM algorithms have been developed for handling multi-class problems; of these, one approach is to use a combination of several binary SVM classifiers, such as one-versus-all (Vapnik, 1998), one-versus-one (Kressel, 1999), and the directed acyclic graph (DAG) SVM (Platt et al., 2000), while another approach is to directly use a single optimization formulation (Crammer and Singer, 2001). Owing to its computational expensiveness and complexity, single SVM formulation is usually avoided. The multi-classification of samples with n classes can be considered as constructing and combining several binary categorization problems. The earliest approach for multiclassification using SVM was one-versus-all. Multi-classification with this method can be described as the task of constructing n binary SVMs. The ith SVM is trained with the samples from the ith class positive, and the samples from all the other classes negative. N classification decision functions can be found: f i (x) =

l

y ij αij k(x ij , x) + bi , i = 1, . . . , n

(4.41)

j=1

where yji ∈ {+1, −1}, k is a kernel function, bi is a bias term, and αij is the coefficient obtained by solving a convex quadratic programming problem. Given an unknown sample (denoted by x), the input vector x is assigned to the class that has the largest value of the decision function in equation (4.41).


Support vectors of class +1

Support vectors of class −1

(a)

(b)

(c) Figure 4.8 classifier.

Performance of (a) a linear SVM classifier; (b) a polynomial SVM classifier; (c) an RBF SVM


Another approach using a combination of several binary SVM classifiers is called the one-versus-one method. Multi-classification with this method can be described as the task of constructing n(n − 1)/2 binary SVMs, one classifier C ij for every pair of distinct classes, i.e. the ith class and the jth class, where i = j, i = 1, . . . , n; j = 1, . . . , n. Each classifier C ij is trained with the samples in the ith class with positive labels, and the samples in the jth class with negative labels. The classification decision functions can be constructed as detailed below: f (x) = ij

sum

ij ij

ij

y k αk k(x k , x) + bij ,

i = j, i = 1, . . . , n; j = 1, . . . , n

(4.42)

k=1

where the sum is the total number of the ith and jth classes from the training data, ij ij yk ∈ {+1, −1}, k is a kernel function, bij is a bias term, and αk is the coefficient obtained by solving a convex quadratic programming problem. Given an unknown sample, if the decision function in equation (4.42) states that the input vector x is in the ith class, the classifier C ij casts one vote for the ith class; otherwise the vote for the jth class is added by one. When all the votes from the n(n − 1)/2 classifiers are obtained, the unknown sample x is assigned to the class with the most votes. The third approach is the directed acyclic graph SVM, which is a learning algorithm designed by combining many two-class classifiers into one multi-class classifier using a decision graph. The training phase of the multi-classification is the same as the oneversus-one method, i.e. it constructs n(n − 1)/2 binary classifiers. However, in the test phase it utilizes a new multi-class learning architecture called the decision directed acyclic graph (DDAG). Each node of the DDAG associates with a one-versus-one classifier. Supposing there are five categories in the samples, Figure 4.9 illustrates the

1/5

1, 2, 3, 4, 5 1

2/5

2, 3, 4, 5

1, 2, 3, 4

2

3/5

3, 4, 5

4/5

5

5

5

3, 4

4

1/4 4

1

2/4

2, 3, 4

3

4, 5

5

3/4

2, 3

3

1

4

2

1/3

1, 2, 3

2/3

3

Figure 4.9 The DDAG for classification of samples with five categories.

1, 2

2

1/2

1


DDAG procedure of multi-classification. Given an unknown sample x, first the binary decision function at the root node is evaluated. Then, if the value of the binary decision function is −1, the node exits via the left edge; otherwise, if the value is +1, via the right edge. Similarly, the binary decision function of the next internal node is then evaluated. The class of x is the one associated with the final leaf node.

7 Conclusions A number of classification techniques have been introduced in this chapter, including the artificial neural network, Bayesian classification, discriminant analysis, nearest neighbor, fuzzy logic, the decision tree, and the support vector machine. All of the above methods have shown feasibility for the classification of food products, with various successes. Given the proliferation of classification techniques, it is not an easy task to select an optimal method that can be applied to different food products. It is impossible to offer one technique as a general solution because each classification technique has its own strengths and weaknesses and is suitable for particular kinds of problem. As a result, one of the most interesting fields for further application is to combine several techniques for classification of food products. Another trend for further application is to adopt relatively novel classification techniques, such as SVM.

Nomenclature ij

αi , αij , αn δ w γ µ ω σ τ(x) ϕ(·) λξi η A b, bi , bij c1 , c2 , . . . , cn c C ij c(k) CMB CMW CR

coefficient obtained by solving a quadratic programming problem error between the actual class and predicted class delta weight learning rate coefficient mean vector normal to the hyperplane sigma term of Gaussian radial basis function kernels membership function non-linear transformation parameter used to penalize variables ξi slack variables probability of the bound holding prior assumptions and experience bias term classes from number 1 to n desired output class classifier for the ith class and the jth class class of the kth training pattern between-class sample covariance matrix within-class and sample covariance matrix classification risk

Nomenclature 103

DC DE DM [o] ej ERemp f G gain(xk , TS) gainratio(xk , TS) i, j, k, n J (w) J (W ) k(xi , xj ) l m M MB MW P(ci |A) P(x|A) P(ci |xA) P(x|ci A) S S[s] j split(xk , TS) tf U wi w W [s] wji x, x1 , x2 x1 , x2 , . . . , xd x X X y y[o] [s] yj

city-block distance Euclidean distance Mahalanobis distance local error empirical risk classification decision function matrix of class membership matrix information gained on feature xk ratio between the information gained and the information due to the split of TS indices ratio of between-class variance to within-class variance ratio of the determinants of the within-class to the between-class covariance matrices kernel function number of samples in a training set degree of polynomial kernels matrix of class means between-class scatter matrix within-class scatter matrix prior probability of class ci conditional probability to the prior assumptions and experience A posterior probability class-conditional probability density for observation x in class ci and the prior assumptions and experience A summation of weighted input summation of weighted inputs to jth processing element in layer s information due to the split of TS on the basis of the value of feature xk transfer function means of all the group of samples arc weight weight vector projection matrix connection weight joining ith processing element in layer (s − 1) to jth processing element in layer s fuzzy variables features from number 1 to d sample feature vector fuzzy space matrix of all the group of samples output class output vector output state of jth processing element in layer s


Abbreviations: AL ANN DAG E HDFS I k-NN PE S SRM SVM TS UL VCD

acceptable level artificial neural network directed acyclic graph global error function high-dimensional feature space information needed to identify the class of an element k-nearest-neighbor processing element fuzzy set structural risk minimization support vector machine training set unacceptable level Vapnik Chervonenkis dimension

References Abdullah MZ, Aziz SA, Mohamed AMD (2000) Quality inspection of bakery products using a color-based machine vision system. Journal of Food Quality, 23 (1), 39–50. Abdullah MZ, Guan LC, Lim KC, Karim AA (2004) The applications of computer vision system and tomographic radar imaging for assessing physical properties of food. Journal of Food Engineering, 61 (1), 125–135. Aleixos N, Blasco J, Navarrón F, Moltó E (2002) Multispectral inspection of citrus in realtime using machine vision and digital signal processors. Computers and Electronics in Agriculture, 33 (2), 121–137. Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Mining Knowledge Discovery, 2 (2), 1–43. Chao K, Chen Y-R, Hruschka WR, Gwozdz FB (2002) On-line inspection of poultry carcasses by a dual-camera system. Journal of Food Engineering, 51 (3), 185–192. Chtioui Y, Panigrahi S, Backer LF (1999) Rough sets theory as a pattern classification tool for quality assessment of edible beans. Transactions of the ASAE, 42 (4), 1145–1152. Crammer K, SingerY (2001) On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2, 265–292. Devroye L, Györfi L, Lugosi G (1996) A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag. Domenico S, Gary W (1994) Machine vision and neural nets in food processing and packaging – natural way combinations. In Food Processing Automation III – Proceedings of the FPAC Conference, ASAE, Orlando, Florida, USA. Du C-J, Sun D-W (2004) Shape extraction and classification of pizza base using computer vision. Journal of Food Engineering, 64 (4), 489–496. Du C-J, Sun D-W (2005a) Pizza sauce spread classification using color vision and support vector machines. Journal of Food Engineering, 66 (2), 137–145.

References 105

Du C-J, Sun D-W (2005b) Comparison of three methods for classification of pizza topping using different color spaces. Journal of Food Engineering, 68 (3), 277–287. Dutta S (1993) Fuzzy Logic Applications: Technological and Strategic Issues. INSEAD (European Institute of Business Administration), Boulevard de Constance, 77305 Fontainebleau Cedex, France. Fisher RA (1936) The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179–188. Ghazanfari A, Wulfsohn D, Irudayaraj J (1998) Machine vision grading of pistachio nuts using gray-level histogram. Canadian Agricultural Engineering, 40 (1), 61–66. Hart PE (1968) The condensed nearest neighbour rule. IEEE Transactions on Information Theory, 14, 515–516. Howarth MS, Searcy SW (1992) Inspection of fresh market carrots by machine vision. In Food Processing Automation II – Proceedings of the 1992 Conference, ASAE, Lexington Center, Lexington, Kentucky, USA. Jahns G, Nielsen HM, Paul W (2001) Measuring image analysis attributes and modelling fuzzy consumer aspects for tomato quality grading. Computers and Electronics in Agriculture, 31, 17–29. Kavdir I, Guyer DE (2002) Apple sorting using artificial neural networks and spectral imaging. Transactions of the ASAE, 45 (6), 1995–2005. Kressel UH-G (1999) Pairwise classification and support vector machines. In Advances in Kernel Methods: Support Vector Learning (Schölkopf B, Burges CJC, Smola AJ, eds.). Cambridge: MIT Press, pp. 255–268. Leemans V, Destain M-F (2004) A real-time grading method of apples based on features extracted from defects. Journal of Food Engineering, 61 (1), 83–89. Li J, Tan J, Shatadal P (2001) Classification of tough and tender beef by image texture analysis. Meat Science, 57, 341–346. Li QZ, Wang MH, Gu WK (2002) Computer vision based system for apple surface defect detection. Computers and Electronics in Agriculture, 36 (2–3), 215–223. Luo X, Jayas DS, Symons SJ (1999) Comparison of statistical and neural network methods for classifying cereal grains using machine vision. Transactions of the ASAE, 42 (2), 413–419. Majumdar S, Jayas DS (2000a) Classification of cereal grains using machine vision: I. Morphology models. Transactions of the ASAE, 43 (6), 1669–1675. Majumdar S, Jayas DS (2000b) Classification of cereal grains using machine vision: II. Color models. Transactions of the ASAE, 43 (6), 1677–1680. Majumdar S, Jayas DS (2000c) Classification of cereal grains using machine vision: III. Texture models. Transactions of the ASAE, 43 (6), 1681–1687. Martens H, Martens M (2001) Chapter 6. Analysis of two data tables X and Y: Partial Least Squares Regression (PLSR). In Multivariate Analysis of Quality: an Introduction. London: John Wiley & Sons, pp. 111–125. Michie D (1991) Methodologies from machine learning in data analysis and software. The Computer Journal, 34 (6), 559–565. Mitchell RS, Sherlock RA, Smith LA (1996) An investigation into the use of machine learning for determining oestrus in cows. Computers and Electronics in Agriculture, 15 (3), 195–213.


Mitchell T (1997) Machine Learning. New York: McGraw-Hill. Nagata M, Cao Q (1998) Study on grade judgment of fruit vegetables using machine vision. Japan Agricultural Research Quarterly, 32 (4), 257–265. Okamura NK, Delwiche MJ, Thompson JF (1993) Raisin grading by machine vision. Transactions of the ASAE, 36 (2), 485–492. Paliwal J, Visen NS, Jayas DS (2001) Evaluation of neural network architectures for cereal grain classification using morphological features. Journal of Agricultural Engineering Research, 79 (4), 361–370. Pardo M, Sberveglieri G (2002) Support vector machines for the classification of electronic nose data. In Proceedings of the 8th International Symposium on Chemometrics in Analytical Chemistry, Seattle, USA. Park B, Lawrence KC, Windham WR, Chen Y-R, Chao K (2002) Discriminant analysis of dual-wavelength spectral images for classifying poultry carcasses. Computers and Electronics in Agriculture, 33 (3), 219–231. Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In Proceedings of Neural Information Processing Systems. Cambridge: MIT Press, pp. 547–553. Quinlan JR (1993) C4.5: Programs for Machine Learning. San Mateo: Morgan Kauffman Publishers. Rao C, Mitra S (1971) Generalized Inverse of Matrices and Its Applications. New York: John Wiley & Sons. Rao CR (1948) The utilization of multiple measurements in problems of biological classification (with discussion). Journal of the Royal Statistical Society, Series B, 10, 159–203. Rousu J, Flander L, Suutarinen M, Autio K, Kontkanen P, Rantanen A (2003) Novel computational tools in bakery process data analysis: a comparative study. Journal of Food Engineering, 57 (1), 45–56. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In Parallel Data Processing, Vol.1 (Rumelhart D, McClelland, J, eds). Cambridge: MIT Press, pp. 318–362. Shahin MA, Tollner EW, Evans MD, Arabnia HR (1999) Watercore features for sorting Red Delicious apples: a statistical approach. Transactions of the ASAE, 42 (6), 1889–1896. Shahin MA, Tollner EW, McClendon RW (2001) Artificial intelligence classifiers for sorting apples based on watercore. Journal of Agricultural Engineering Research, 79 (3), 265–274. Shahin MA, Tollner EW, Gitaitis RD, Sumner DR, Maw BW (2002) Classification of sweet onions based on internal defects using image processing and neural network techniques. Transactions of the ASAE, 45 (5), 1613–1618. Song YH, Kim SJ, Lee SK (2002) Evaluation of ultrasound for prediction of carcass meat yield and meat quality in Korean native cattle (Hanwoo). Asian Australasian Journal of Animal Sciences, 15 (4), 591–595. Sonka M, Hlavac V, Boyle R (1999) Image Processing, Analysis and Machine Vision. El Dorado Hills: PWS Publishing. Storbeck F, Daan B (2001) Fish species recognition using computer vision and a neural network. Fisheries Research, 51 (1), 11–15.

References 107

Trihaas J, Bothe HH (2002) An application of support vector machines to E-nose data. In Proceedings of the 9th International Symposium on Olfaction & Electronic Nose, Rome, Italy. Vapnik V (1995) The Nature of Statistical Learning Theory. New York: Springer-Verlag. Vapnik V (1998). Statistical Learning Theory. New York: John Wiley & Sons. Vízhányó T, Felföldi J (2000) Enhancing color differences in images of diseased mushrooms. Computers and Electronics in Agriculture, 26 (2), 187–198. Wilson D (1972) Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man and Cybernetics, 2, 408–421. Zadeh L (1965) Fuzzy sets. Information and Control, 8, 338–353. Zayas I, Converse H, Steele J (1990) Discrimination of whole from broken corn kernels with image analysis. Transactions of the ASAE, 33 (5), 1642–1646.

Quality Evaluation of Meat Cuts Liyun Zheng1 , Da-Wen Sun1 and Jinglu Tan2 1 Food Refrigeration and Computerised Food Technology,

University College Dublin, National University of Ireland, Dublin 2, Ireland 2 Department of Biological Engineering, University of Missouri, Columbia, MO 65211, USA

1 Introduction Currently meat quality is evaluated through visual appraisal of certain carcass characteristics, such as marbling (intramuscular fat), muscle color, and skeletal maturity. Although the visual appraisal method has been serving the meat industry for many years, the subjective evaluation leads to some major intrinsic drawbacks, namely inconsistencies and variations of the results in spite of the fact that the graders are professionally trained (Cross et al., 1983). This has seriously limited the ability of the meat industry to provide consumers with products of consistent quality, and subsequently its competitiveness. As there is always a desire from the meat industry for objective measurement methods, many research efforts have been devoted to developing instruments or devices. One obvious and popular approach is to measure the mechanical properties of meat as indicators of tenderness, with the most well known perhaps being the Warner-Bratzler shear-force instrument. For cooked meat, the shear strength correlates well with sensory tenderness scores (Shackelford et al., 1995); however, such a method is not practical for commercial fresh-meat grading. To overcome this problem, one of the most promising methods for objective assessment of meat quality from fresh-meat characteristics is to use computer vision (Brosnan and Sun, 2002; Sun, 2004). Recently, applications of computer vision for food quality evaluation have been extended to food in many areas, such as pizza (Sun, 2000; Sun and Brosnan, 2003a, 2003b; Sun and Du, 2004; Du and Sun, 2005a), cheese (Wang and Sun, 2002a, 2002b, 2004), and cooked meats (Zheng et al., 2006a; Du and Sun, 2005b, 2006a, 2006b). However, for fresh meats, research began in the early 1980s. For example, Lenhert and Gilliland (1985) designed a black-and-white (B/W) imaging system for lean-yield estimation, and the application results were reported by Cross et al. (1983) Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


5

112 Quality Evaluation of Meat Cuts

and Wassenberg et al. (1986). Beef quality assessment by image processing started with the work by Chen et al. (1989) to quantify the marbling area percentage in six standard USDA marbling photographs, and later on McDonald and Chen (1990a, 1990b) used morphological operations to separate connected muscle tissues from the longissimus dorsi (LD) muscle. For quality evaluation of other fresh meat, such as pork and lamb, early studies were performed by Kuchida et al. (1991) and Stanford (1998). The composition (fat and protein %) of pork were analyzed based on color video images (Kuchida et al., 1991) and video-image analysis was also used for on-line classification of lamb carcasses (Stanford, 1998). Since then, research has been progressing well in this area. To develop a computer vision system (CVS) for objective grading of meat quality, several steps are essential. Although the existing human grading system has many intrinsic drawbacks, any new systems designed as a replacement must still be compared with the human system before they can be accepted. Furthermore, the existing human grading system is qualitative, whereas the quantitative characteristics that contribute to the human grading are not always obvious. Therefore, it is necessary to search for image features that are related to human scores for marbling abundance, muscle color, and maturity – and, eventually, official grades such as USDA grades. Moreover, to improve the usefulness of the grading system, new instrumentally-measurable characteristics are needed to enhance the power of the grades in predicting eating quality, such as tenderness.

2 Quality evaluation of beef 2.1 Characterizing quality attributes Meat images can be processed by computer vision to characterize quality attributes such as those defined in the Japanese Beef Marbling Standard and in the USDA beef grading system. Color-image features have been extracted to predict human scores of color, marbling, and maturity (Tan, 2004). Studies have also been conducted to predict the Japanese Beef Color Standard (BCS) number based on beef images (Kuchida et al., 2001a). 2.1.1 Color and marbling

Computer vision technique has been demonstrated as a rapid, alternative, and objective approach for measuring beef color and marbling. The pioneering work in this area was conducted by McDonald and Chen (1990a, 1990b, 1991, 1992). Based on reflectance characteristics, fat and lean in the longissimus dorsi (LD) muscle were discriminated to generate binary muscle images (McDonald and Chen, 1990a, 1990b, 1991, 1992). A certain degree of correlation between total fat surface area and sensory panel scores for marbling was achieved, with an r 2 of 0.47. Data from these early studies also suggest that it is not reliable to measure only the visible total fat area to distinguish multiple categories of marbling (McDonald and Chen, 1991). In order to improve marbling score prediction, McDonald and Chen (1992) later proposed the use of a Boolean random

Quality evaluation of beef 113

model to describe the spatial marbling distribution, and made significant improvements in prediction accuracy. In Japan, Kuchida and his co-workers (1992, 1997a, 1997b, 1998, 2000a, 2000b, 2001a, 2001b, 2001c) conducted a series of studies in using computer vision to determine marbling scores of beef. Kuchida et al. (1997a) used images of ribeye muscle from 16 Japanese Black steer carcasses and various models of beef marbling standards (BMS 2-12) to assess fat as a percentage of ribeye muscle, the number of intramuscular fat deposits (marblings), and the characteristics (area and form coefficient) of each marbling. The BMS (Beef Marbling Standard) used was developed by the Japanese National Institute of Animal Industry for the evaluation of beef marbling in 1988. Kuchida et al. (1997a) showed that, in ribeye muscle, the fat percentage as determined by computer vision correlated highly with the marbling score as determined visually (r = 0.87, P < 0.01). In order to establish correlation between fat area ratio and marbling, Kuchida et al. (2000a) used the marbling degree of semispinalis capitis (SC) and semispinalis dorsi (SD) muscles from cattle as supplementary parameters in the BMS to evaluate images of 99 cross-sections of SC, SD and LD muscles. It was shown that image features of cross-sections of the ribeye muscle had the potential for prediction of crude fat content (Kuchida et al., 2000b). Ratios of fat over lean area in longissimus dorsi muscles from 22 Japanese Black cattle (steers) were also determined in their studies (Kuchida et al., 2001c). In order to improve the results, a stepwise multiple regression with the BMS value assigned by a grader as the dependent variable was conducted, using 148 independent covariates. It was shown that the BMS value could be predicted reasonably accurately by multiple regression with six covariates selected by the stepwise approach (Kuchida et al., 2001b). In addition, similar work was also conducted by Ishii et al. (1992) and Ushigaki et al. (1997). The suitability of using the BMS value for evaluating beef marbling was compared with that of the marbling score. Based on the relationship between the BMS value (or marbling score) and the ratio of the intramuscular fat area over the total ribeye area (determined by image analysis), it was found that the BMS value is a more appropriate scale than the marbling score for evaluating beef marbling. In the USA, the color-image processing technique has also been applied to the assessment of muscle color and marbling scores of beef ribeye steaks. Tan (2004) extracted color-image features to predict human scores of color, marbling, and maturity. Sixty whole beef ribs representing various marbling and color scores were obtained from a local supplier, and 5-cm thick slices were taken from the samples. Each slice was then cut into two 2.5-cm thick samples. The two freshly cut opposite steak surfaces were used for analysis; one for image acquisition and the other for sensory analysis. Besides the sensory evaluations, a color-image system was also used to capture sample images, with the illumination and camera settings carefully selected to feature an appropriate resolution for revealing the small marbling flecks. The captured images were then subject to a series of processing techniques: image filter, background removal, segmentation of fat from muscles, isolation of the LD muscles, and segmentation of marbling from the LD muscle (Gao et al., 1995; Lu and Tan, 1998; Lu, 2002). Figure 5.1 shows an original image and the corresponding segmented one. The holes in the LD muscles give the image of marbling flecks.


(a)

(b) Figure 5.1 Beef image: (a) original; (b) segmented LD muscle. The holes in the LD muscle give the marbling image.

Features relating to the color of the entire LD muscle were extracted from the muscle images. The LD muscle color was characterized by the means (µR , µG , and µB ) and standard deviations (σR , σG , and σB ) of the red, green, and blue color components. Features representing the amount and spatial distribution of marbling were extracted, and the size of each marbling fleck was also calculated. To account for the effects of fleck size, marbling was classified into three categories according to area: A1 < 2.7 mm2 , A2 = 2.7–21.4 mm2 and A3 > 21.4 mm2 . Several marbling features were computed to measure the marbling abundance: Dci (number of marbling flecks in size category


Ai per unit ribeye area), Dai (sum of marbling area in size category Ai per unit ribeye area), Dc (number of all marbling flecks per unit ribeye area), and Da (total marbling area per unit ribeye area). Marbling was rated by a 10-member trained panel according to the USDA marbling scorecards in a 9-point scale where 1 = devoid, 2 = practically devoid, 3 = traces, 4 = slight, 5 = small, 6 = modest, 7 = moderate, 8 = slightly abundant, and 9 = moderately abundant. Color was evaluated according to a beef color guide in an 8-point scale: 1 = bleached red, 2 = very light cherry red, 3 = moderately light cherry red, 4 = cherry red, 5 = slightly dark red, 6 = moderately dark red, 7 = dark red, and 8 = very dark red. The panel averages were used as the sensory scores. The results indicated that blue mean (µB ) was not significant for color prediction, whereas red and green (µR and µG ) were significant. This suggests that although all three color components varied, the green component might not affect the panelists’ scoring. The fact that µR was significant in marbling prediction showed that the judges’ opinions were influenced by the lean color. Results also showed that both the count and area densities of small marbling flecks (Dc1 and Da1 ) influenced the sensory scores, which was expected. The area density of large flecks (Da3 ) was also significant in influencing the sensory scores, indicating that the panelists were easily influenced by the presence of a few large marbling flecks, although in the sensory analysis they were instructed not to put more weight on larger flecks. Therefore, the global marbling area density (Da ) was influential in the scoring. Statistical analysis indicated that µR and µG were significant for color scores, while µR , Dc1 , Da1 , Da3 , and Da were useful for marbling scores. The r 2 values of regression were 0.86 and 0.84 for color and marbling, respectively, showing the usefulness of the above features in explaining the variations in sensory scores. The above study (Tan, 2004) shows that the image features characterizing the spatial variation of marbling are not significant in the regression. McDonald and Chen (1992) also indicated that information on the spatial distribution of marbling does not correlate significantly with marbling scores. In order to improve the results, Tan et al. (1998) used fuzzy logic and artificial neural network techniques to analyze the sensory scores. In this study (Tan et al., 1998), the fuzzy sets, fuzzy variables, and sample membership grades were represented by the sensory scales, sensory attributes, and sensory responses, respectively. Multijudge responses were formulated as a fuzzy membership vector or fuzzy histogram of response, which gave an overall panel response free of unverifiable assumptions implied in conventional approaches. Then, from the image features selected by backward elimination, neural networks were employed to predict the sensory responses in their naturally fuzzy and complex form. Finally, a maximum method of defuzzification was used to give a crisp grade of the majority opinion. With this improvement by using the fuzzy set and neural network, a 100 percent classification rate was achieved for the color and marbling, which further verified the usefulness of the image features extracted. The artificial neural network technique was also used to enhance the robustness of a hybrid image-processing system which can automatically distinguish lean tissues in beef cut images with complex surfaces, thus generating the lean tissue contour (Hwang


et al., 1997). Furthermore, Subbiah (2004) also developed a fuzzy algorithm to segment fat and lean tissue in beef steaks (longissimus dorsi). The fat and lean were differentiated using a fuzzy c-means clustering algorithm using convex hull procedures. The LD was segmented from the steak using morphological operations of erosion and dilation. After each erosion–dilation iteration, a convex hull was fitted to the image to measure the compactness. Iterations were continued, to yield the most compact LD. The algorithm has been found to segment the steaks with a classification error of 1.97 percent. Computer vision was also tested by Dasiewicz et al. (2002) to analyze the color of LD from 30 dairy and beef cattle. A significant correlation was found between either texture and CIE-Lab color features or texture and the pH values, regardless of the meat type (dairy and beef). This study confirmed the advantage of using computer-image analysis as a tool for evaluating chemical composition and marbling characteristics of beef cuts. 2.1.2 Skeletal maturity

The USDA beef grading system uses the lean color and the degree of cartilage ossification at the tips of the dorsal spine of the sacral, lumbar, and thoracic vertebrae to determine the physiological maturity of beef carcasses. Such an evaluation is subjective and prone to human biases in spite of the professional training received by the graders. Therefore, in order to improve the consistency of results and obtain more precise description of products, objective measurements of the physiological maturity of cattle carcasses is desirable. The computer vision technique is one such objective method. In a study conducted by Hatem and Tan (1998), color images from 110 beef carcass with USDA maturity scores ranging from “A” (young) to “E” (old) were taken. For “A” maturity the cartilage in the thoracic vertebrae is free of ossification, and for “B” maturity there is some evidence of ossification. Then, the cartilage becomes progressively ossified with age until it appears as bone. As the degree of cartilage ossification in the vertebrae is the most important indicator of skeletal maturity, only images focused on the thoracic vertebra around the thirteenth to fifteenth ribs were taken (Figure 5.2). The images were initially segmented to isolate the bone from the cartilage using color and spatial features (Hatem and Tan, 2000, 2003; Hatem et al., 2003). The hue component in the HSI (hue, saturation, and intensity) color space was found to be effective in segmenting the cartilage areas, while the component in the CIE-Lab color space gave good results for segmenting the bones. A set of morphological operations was conducted to refine and combine the segmented cartilage and bone into a bone–cartilage object, which was then used to characterize the degree of cartilage ossification. Compared with bone, the color of cartilage is normally lighter. Therefore, color varies along the bone–cartilage differently due to different degrees of ossification (Hatem and Tan, 1998). For animals with “A” maturity, which have more cartilage and thus give a longer segment of light colors along the length of the carcass, the average hue values calculated along the length of the bone–cartilage object are useful image features. Therefore, Hatem and Tan (1998) used these hue values as input vectors to a neural network and the maturity score as the output of the network. The network was trained by using the back-propagation algorithm. The trained neural network could then be used as maturity score predictor. For every set of samples from the 110 beef carcasses, it was divided into five subsets – four for training and the fifth for testing, in a rotating


(a)

(b) Figure 5.2 Image of vertebrae: (a) original; (b) outline of bone–cartilage objects.

manner. The fuzzy set technique (Tan et al., 1998) was incorporated in the use of scoring by the professional grader. The maturity scores predicted by the neural network were compared with the human scores to calculate the correct classification rate. Results show that the average rates for the five rotations varied from 55 to 77 percent (Tan et al., 1998). The above algorithm was applied to another group of samples of 28 cattle of known age, most of which were of “A” maturity while the rest were of “B” maturity. An average 75 percent classification rate was obtained, indicating the generality and robustness of the above procedure (Hatem et al., 2003).

2.2 Predicting quality and yield grades Generally speaking, beef carcasses are yield-graded by visual appraisal of the twelfth rib surface and other parts of a carcass. In the USA, USDA standards are used


Figure 5.3

Fat area for the beef sample shown in Figure 5.1.

for the grading of the carcass – i.e., carcass yield (lean percentage) is determined by considering (1) the amount of external fat, or the fat thickness over ribeye muscle; (2) the amount of kidney, heart, and pelvic fat; (3) the area of the ribeye muscle, and (4) the carcass weight (Lu and Tan, 1998). Computer vision has been investigated as a tool to achieve the above grading (Lu et al., 1998; Soennichsen et al., 2005, 2006). In an early study conducted by Lu et al. (1998), beef carcasses (247 for quality grading, 241 for yield grading) of the same maturity were selected in a commercial packing plant and prepared according to normal industrial practice. Each carcass was graded by an official USDA grader with an eight-point scale (prime, choice, select, standard, commercial, utility, cutter, and canner) for quality, and a five-point scale (1 to 5) for yield. Immediately after the official grading, digital color images of the ribbed surfaces (steaks) were captured under constant lighting conditions. The images went through various steps of segmentation to obtain the regions of interest and to extract relevant color and marbling. Figure 5.3 shows an example of the extracted fat area image processed based on the image shown in Figure 5.1. As fat thickness is an important indicator of lean meat yield, the back-fat area was partitioned into the dorsal part (the upper-left half of the fat area in Figure 5.3) and the ventral part (the lower-right half of the fat area in Figure 5.3). The thickness was then computed in the direction approximately perpendicular to the back curvature (lower boundary of the fat area in Figure 5.3), with the average thickness for both parts being used as the fat thickness. Divergence maximization using linear and non-linear transforms was employed to maximize the differences among classes (Lu and Tan, 1998). For quality classification, only linear transforms were applied; for yield classification, linear quadratic and cubic transforms were employed. Supervised classifiers were trained for both quality and yield classification. The data set was randomly partitioned into ten subsets, nine of them for training and the tenth for testing, in a rotating fashion until each of the ten subsets was tested.


For quality classification, the correct rate varied with the rotations of the procedure. For a total of ten rotations, three were 100 percent; four were 90–99 percent, and the remaining three were 60–70 percent. Therefore, the average rate was 85.3 percent. For yield classification, the correct rate was above 50 percent for eight out of the ten rotations with the linear transform. Using quadratic and cubic transforms did not significantly improve the correct rate. The linear transform yielded the best performance, with an average rate of 64.2 percent. The quality classification result was considered excellent, while the yield result was reasonably good. Cannell et al. (2002) employed a dual-component computer vision system (CVS) to predict commercial beef subprimal yields and to enhance USDA yield grading. In the system, the first video camera captures an image of the outside surface and contour of unribbed beef, while the second records an image of the exposed twelfth/thirteenth rib interface after ribbing. Before the carcasses from 296 steer and heifer cattle were cut into industry-standard subprimal cuts, the carcasses were evaluated by the CVS and by USDA official graders and on-line graders. The results indicated that the CVS predicted wholesale cut yields more accurately than did the on-line yield grading. When the estimated ribeye area was replaced by the computer vision measurement in determination of USDA yield grade, accuracy of the cutability prediction similar to that of USDA official graders was achieved. The dual-component CVS was also employed by Steiner et al. (2003) to enhance the application of USDA yield grade standards at commercial chain speeds for cattle carcasses. The system measured the longissimus muscle area of carcasses at the twelfth/thirteenth rib interface and combined the measured data with on-line grader estimates of yield grades, resulting in an increase in the accuracy of yield grade prediction. In a separate study, Shackelford et al. (2003) used a specially developed image analysis system for on-line prediction of the yield grade, longissimus muscle area, and marbling score of 800 cattle carcasses at two beef-processing plants. Prediction equations developed incorporating hot carcass weight and image features could account for 90, 88 and 76 percent of variation in calculated yield grade, longissimus muscle area, and marbling score, respectively. As comparison, official USDA yield grade as applied by on-line graders was able to account for only 73 percent of variation. Therefore the system had the potential for improving accuracy of yield grade determination; however, it could not accurately predict the marbling. BeefCam is a video-imaging technology that scans beef carcasses into colordifferentiated images from which the subsequent eating quality can be predicted. For instance, BeefCam can be used to measure lean color as an indicator of beef tenderness, since the color relates to the pH values of the lean tissue. Wyle et al. (2003) tested the prototype BeefCam system to sort cattle carcasses into expected palatability groups. The system was either used alone or in combination with USDA quality grades assigned by line-graders. A total of 769 carcasses from four commercial, geographically dispersed beef packing plants were used. These carcasses were divided into three USDA quality groups – Top Choice, 241 carcasses; Low Choice, 301 carcasses; Select, 227 carcasses. Before each use, the system was calibrated with a white standard card. Images of longissimus muscles at the twelfth/thirteenth rib interface were then captured. These images were processed and analyzed using two regression models: one


only used BeefCam data while the other also used a coded value for quality grade. These two models were validated with 292 additional carcasses at another plant. The quality data were also obtained as determined by Warner-Bratzler shear force after 14 days of aging and sensory measurements on corresponding cooked strip loin steaks. Results confirmed the usefulness of the BeefCam system, as sorting by BeefCam reduced the number of carcasses in the “certified” group, which generated steaks of tough or unacceptable overall palatability. Research was also conducted in Europe to study the capability of CVS for grading carcasses according to the official EUROP scheme (EC 1208/1981). Soennichsen et al. (2005, 2006) applied image analysis to grade 1519 calf carcasses. The CVS predicted accurately the fat class on a 15-point scale (EUROP grid with subclasses); however, its accuracy was poorer for conformation, suggesting that a special scale was needed for calf carcasses. The system also predicted the weight of retail cuts with high accuracy, with the residual estimation error of primal cuts and retail cuts being 1.4–5.2 percent. Prediction of the total and the saleable meat weight was also very accurate, with residual estimation errors of 2.1 and 3.5 percent, respectively.

2.3 Predicting carcass composition Early studies using image features to predict beef carcass composition such as lean, fat, and bone can be traced back to the late 1990s. Karnuah et al. (1999, 2001) established equations for predicting cattle-carcass percentages of total lean, total fat, and total bone composition, using data collected from 73 Japanese Black steers slaughtered at 27–40 months of age. The composition data were fitted into various multiple linear regression equations. Correlation coefficients between predicted values and actual values obtained on dissection for weight of lean, fat, and bone were 0.70–0.72, whereas those for percentages of lean, fat, and bone were much lower (0.29–0.63). Anada and Sasaki (1992) and Anada et al. (1993) analyzed the fifth/sixth rib crosssection of beef carcasses to measure the areas of lean, fat and bone, and their total. The dimensions of the longissimus and trapezius muscles, and the distance between the centers of gravity of these two muscles were also measured. A stepwise regression analysis was used to select the best regression equations to predict carcass composition (as weight and percentage of lean, fat, and bone). The total area or fat area was the best predictor for percentage lean; percentage fat area gave the best prediction for fat or bone percentage; while the distance between the centers of gravity of the two muscles was an important predictor for weight of fat and bone. Karnuah et al. (1994) also measured beef composition using fifth/sixth rib cross-sections. Images from 28 fattened cattle were captured to measure individual muscle area, circumference, length of long and short axes, total cross-sectional area, total muscle area, total fat area, total bone area, eccentricity, direction of long axis, and distance between the centers of gravity of any two muscles. Results indicated that excellent repeatability measurements were achieved in using the eccentricity and direction of long axis, total area, total muscle area, total fat area, and total bone area of the carcass cross-section for the prediction of carcass composition.


Images of cross-sections cut at other locations in beef carcasses were also used to predict composition. Nade et al. (2001) used images from cross-sectional ribloin cut between the sixth and seventh rib bones of 24 Japanese Black cattle (steer) carcasses. Predictive equations were derived for estimating composition parameters such as total area, muscle area, fat area, ratio of muscle to fat, and shape of the longissimus and trapezius muscles. The actual weight and ratio of muscle to fat were determined through physical dissection from the left side of the carcass. The ribeye area, ratio of muscle to total area, and carcass weight were used to predict the muscle weight. The ribeye area, ratio of fat to total area, and carcass weight were used to estimate the amount of muscle in the carcass, the fat weight and the amount of fat in the carcass. Results indicated that the ribeye area, the ratio of fat to total area, and the carcass weight are important parameters for carcass composition prediction. Lu and Tan (2004) predicted lean yield by measuring the twelfth rib surface of cattle carcasses and compared the CVS results with USDA yield characteristics and USDA yield grades. Different multiple linear regression models were developed for data from each set of measurements on 241 cattle carcasses, and the models were found to be suitable for lean yield prediction. Results also indicated that percentage of ribeye area was a more useful predictor of lean yield than fat thickness. Marbling count and marbling area density were also useful for prediction. However, prediction of lean percentage was not as accurate as that of lean weight.

2.4 Predicting tenderness As discussed previously, marbling and color are two key grades in beef quality, especially for young animals such as those classified as “A” or “B” maturity in the USDA system. However, these two quality factors are weak predictors of meat texture attributes such as tenderness. Meat texture is a measure of the fineness of a cut surface, which is influenced by the size of the muscle fibers and/or muscle-fiber bundles visible on a transversely cut surface. The texture of muscles can vary from a velvety, light structure to a coarse, rough structure, and may also be influenced by the amount of connective tissue and marbling. Therefore, meat surface texture can be a good indicator of tenderness. Research on predicting meat texture is the most challenging of computer vision applications for meat quality evaluation. Fortunately, meat texture can be related to image texture, which is an important characteristic of images. Image texture, usually referred to the fineness, coarseness, smoothness, granulation, randomness or lineation of images, or how mottled, irregular or hummocky images are, can be quantitatively evaluated (Haralick, 1973). For image texture analysis, a variety of techniques are available (Zheng et al., 2006b, 2007), including statistical, structural and spectral approaches (Du and Sun, 2004). Among them, the statistical approach is most commonly used with methods of the gray level co-occurrence matrix (GLCM), the gray level difference method (GLDM) and the gray level run length matrix (GLRM) (Du and Sun, 2004, 2006c). Therefore, in order to find the better quantitative predicators for meat texture attributes, computer vision has been investigated as a tool – for example, Sun and co-workers (Du and Sun, 2005b, 2006a, 2006b; Zheng et al., 2006b) have been using computer vision to predict eating quality attributes of cooked meats. For fresh meat cuts, Li et al. (1999, 2001) characterized muscle texture by image processing,


and used color, marbling, and textural features to predict beef tenderness measured by traditional methods such as Warner-Bratzler shear forces and sensory evaluations. 2.4.1 Correlation with Warner-Bratzler shear force

In the experiments performed by Li et al. (1999), 265 carcasses, all of “A” maturity, were selected to differ in USDA quality grades in a commercial packing plant. A rib section (posterior end) was removed and vacuum-packaged; this was later cut into 2.54-cm thick steaks and cooked for Warner-Bratzler shear-force measurements. Eight cores of 1.27-cm diameter from each cooked steak were removed parallel to the muscle fibers, and sheared with a Warner-Bratzler instrument. The shear force varied from 12.25 to 51.35 N, but the average data were used in analysis. Images of the ribbed surfaces were captured in the plant immediately following quality grading, and segmented into muscle, fat and marbling. Image textural features, based on pixel value, run length, and spatial dependence, were computed as predictors of tenderness, as the image texture of beef muscles surface is directly or indirectly related to tenderness. Figure 5.4 shows differences in image textures of beef samples with varying tenderness. These differences can be measured by image processing (Li et al., 1999). A pixel run is defined as a set of connected pixels in the same row having the same or close intensity values. Textural features can be obtained from the statistics of the pixel runs. Pixel value spatial dependence can be described by the so-called co-occurrence matrix. A total of 14 statistical measures (Haralick et al., 1973) were employed to extract textural features from this matrix. The textural features having the highest correlation with shear force were selected for subsequent analyses. Principal component regression (PCR) was performed to test the improvement in shear-force prediction after the textural features were added to the color and marbling features. PCR consists of principal component analysis (PCA) followed by multiple linear regression (MLR) and partial least squares (PLS). The SAS stepwise procedure (Anon, 2000) was performed to select the variables significant for shear-force prediction. Classification analysis was also used to classify the beef samples. The prediction of shear-force values involves comparison among three groups of quality predictors: color and marbling scores graded by UADA official graders; color and marbling features obtained from images; and color, marbling, and textural features from images. When the first group of features was used to predict shear force, the prediction was very poor (r 2 < 0.05); however, when the second group of features was used, the predictions were slightly improved to r 2 = 0.16, where r 2 is the correlation coefficient. The last group of features yielded the best classification results, with r 2 being 0.18 using PCR and 0.34 using partial least square (PLS). However, the prediction results were still poor. The classification procedure was thus improved by the following procedure. Based on the shear-force values (≤1.71 kg, 1.71 kg–3.09 kg, and ≥ 3.09 kg), the beef samples were segregated into three categories. Among them, 198 samples were used as calibration data and 45 samples were used as test data. The SAS Discriminant procedure (Anon, 2000) with a linear discriminant function was used to classify the samples into a category. The calibration samples could be classified with 76 percent accuracy, and the test samples with 77 percent accuracy.


(a)

(b) Figure 5.4 These saturation images of two samples of different tenderness exhibit different image textures. The upper sample is less tender.

The above results show the possibilities of using combined color, marbling, and textural features to improve the models for shear-force prediction; however, the prediction accuracy is still far from satisfactory. Therefore, color, marbling, and muscle image textures may still not contain sufficient information to define cooked-meat shear force. Nevertheless, the inclusion of textural features brought about significant improvement. Therefore, the image texture of muscles is at least a significant indicator of the mechanical properties of beef (Du and Sun, 2006a). Wheeler et al. (2002) compared the accuracy of three objective systems (a portable prototype BeefCam image analysis system, slice shear-force values, and colorimetry) for identifying beef cuts which can be guaranteed as being tender. Longissimus muscles at the twelfth rib from 708 carcasses were assessed. Steaks were cooked


for sensory analysis and Warner-Bratzler shear-force determination. As indicated by Li et al. (1999), only color features (either by BeefCam or colorimetry) were inadequate in predicting tenderness, and slice shear values were still the accurate method for identifying certifiably tender beef. However, if a BeefCam module was integrated with a CVS (Vote et al., 2003), the CVS/BeefCam reading for longissimus muscle areas correlated well with shear values. CVS/BeefCam loin color values were effective in classifying carcasses into groups which produced steaks of varying shear values, except that CVS/BeefCam fat color values were generally ineffective. 2.4.2 Correlation with sensory tenderness

Image features of beef samples have been investigated to correlate with tenderness as determined by sensory analysis (Tan, 2004). In the study conducted by Li et al. (1999), beef samples were obtained from the short loins of pasture-finished steers and feedlot-finished steers of the same maturity grade. Two sets (97 pieces in each set) of short strip loins were used: one for sensory tenderness evaluation performed by a trained ten-person panel, and the other for image analysis. Images of the beef samples were acquired individually for all the samples under the same conditions. The acquired images were processed. Features were extracted, and 37 of them were selected as predictors for further analysis. Of the 97 beef samples, 72 formed a training subset and the remaining 25 samples were used as a test subset. PCR and PLS were performed to test the improvement in tenderness prediction resulting from adding texture features. The SAS stepwise procedure (Anon, 2000) was then performed to select the significant variables. PCR was applied to all the samples, and results indicated that the r 2 was increased to 0.72 after adding the texture features as compared with 0.30 for using color and marbling features alone. In the PLS analysis, the first 14 factors explaining most of the variations were used for regression. For using only the color and marbling features, the r 2 for the training data set and test data set were 0.35 and 0.17 respectively, which were increased to 0.70 and 0.62 respectively after adding texture features. Similar to shear-force prediction, the above improvements confirmed the significant contribution made by the textural features to beef tenderness prediction. A neural network (NN) model with one hidden layer was also developed. The 14 factors from the PLS analysis were used as inputs, and the tenderness scores as the output. The backpropagation algorithm was employed to train the neural network, and the test data subset was used to test the model. The r 2 for the prediction by NN was 0.70, which is similar to those from PCR and PLS (Li et al., 1999). In a further study carried out by Li et al. (2001), samples from 59 crossbred steers were used, which were divided into “tough” (tenderness score 0.05), which indicates that these attributes are not linearly related to tenderness. The variance extracted by the FGLS method, and the sum entropy, entropy, and difference variance extracted by the GLCM method, are correlated more with the WBS of cooked meats, but have not reached the significant level (P > 0.05). The reason can be attributed to the fact that traditional methods are restricted to the analysis of spatial interactions over relatively small neighborhoods on a single scale. However, the scale is related to the size of textural elements, and should be considered in investigating the relationship between image texture features and the tenderness of cooked meats. With the property of preserving local texture complexity, WT can be applied to extract local texture features and to detect multiresolution characteristics. The local textural characteristics represented by the local variance of wavelet coefficients are useful in differentiating two different regions in an image. For further analysis of the relationships between the selected image texture features and WBS, the partial least squares regression (PLSR) technique was applied in the work of Du and Sun (2006c). As one of the techniques for multivariate regression analysis, PLSR is a hybrid of multiple regression and principal component analysis (PCA) (MacFie and Hedderley, 1993), and can be used to understand the relationship between two data sets by predicting one data set (Y) from the other set (X) (Martens and Martens, 2001). It not only provides solutions for both X and Y variables, but also attempts to find the best solution of X to explain the variation of the Y variable set. The estimated regression coefficients of the predicting model for WBS with three factors (Figure 6.4) show that all of the selected image texture features are positively correlated with WBS, thus having a negative impact on the tenderness of cooked meats (Du and Sun, 2006c). Furthermore, EL2B1 and EL3B1 have the highest relationship with WBS, followed by EL3B4, EL2B4, and EL3B3. The contributions of FDM, EL1B1, EL2B2, EL2B3, and EL3B2 to the prediction of WBS are relatively smaller. In another work, Zheng et al. (2006b) found that it was useful to apply multi-scale approaches (Gabor and WT) for the classification of tough and tender cooked beef joints by image texture analysis. Four different groups of image texture features, i.e. wavelet features (WF), Gabor features (GF), wavelet Gabor features (WGF), and a combination of wavelet features and Gabor features (CWG), were extracted from the

Nomenclature 153

images of cooked beef. After reducing the dimensionality with principal component analysis, the four groups of features were employed to classify the tough and tender beef samples based on the clustering results using a linear discrimination function. WGF was found to perform the best for the classification of beef tenderness, followed by WF and CWG, while GF characterized the tenderness with the least confidence. The error rate of WGF was 29.4 percent, indicating the potential of image texture for determining cooked beef tenderness.

6 Conclusions Computer vision can provide an objective, consistent, and efficient way to evaluate the quality of cooked meats as affected by their manufacturing procedures, including shrinkage measurement, pore characterization, color, and image texture extraction. Further research should investigate the microstructure of cooked meats using a camera with higher magnification or modern microscopy techniques, and the internal structures using ultrasound, magnetic resonance imaging, computed tomography, and electrical tomography techniques. Based on the selected image features, a more powerful mathematical model or algorithm should be developed to predict the physical and chemical quality of cooked meats.

Nomenclature φ eccentric angle of a point on the surface of ham θ polar angle Ax area of each circular cross section perpendicular to the x axis a half of the length L b half of the width W c half of the thickness T D projection area of sample for the integration ELmBn energy of the sub-band at the m-th pyramid level and the n-th orientation band fx gradient in the x direction fy gradient in the y direction i, ˜i indexes li axes of the cooked meat shape n number of boundary point of the contour PM perimeter R radial coordinate

Abbreviation: CLAHE CWG FD FGLS

contrast-limited adaptive histogram equalization combination of wavelet features and Gabor features fractal dimension first-order gray-level statistics

154 Quality Measurement of Cooked Meats

GF GLCM HSI L NN PCA PLSR RGB RLM S T TNP TPT V W WBS WF WGF WT

Gabor features gray-level co-occurrence matrix hue, saturation, and intensity length neural network principal component analysis partial least squares regression red, green, and blue run length matrix surface area thickness total number of pore total processing time volume width Warner-Bratzler Shear wavelet features wavelet Gabor features wavelet transform

References Abramowitz M, Stegun IA (eds) (1972) Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover Books on Advanced Mathematics, New York: Dover. Anon (1997) Western European Meat and Meat Products. London: Datamonitor Europe. Bertram HC, Engelsen SB, Busk H, Karlsson AH, Andersen HJ (2004) Water properties during cooking of pork studied by low-field NMR relaxation: effects of curing and the RN − gene. Meat Science, 66 (2), 437–446. Canny J (1986) A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8 (6), 679–698. Daudin JD, KuitcheA (1996) Modelling of temperature and weight loss kinetics during meat chilling for time variable conditions using an analytical based method – III. calculations versus measurements on pork carcass hindquarters. Journal of Food Engineering, 29, 39–62. DeFreitas Z, Sebranek JG, Olson DG, Carr JM (1997) Carrageenan effects on salt soluble meat proteins in model systems. Journal of Food Science, 62, 539–543. Destefanis G, Barge MT, Brugiapaglia A, Tassone S (2000) The use of principal component analysis (PCA) to characterize beef. Meat Science, 56 (3), 255–259. Du C-J, Sun D-W (2004) Recent developments in the applications of image processing techniques for food quality evaluation. Trends in Food Science & Technology, 15 (5), 230–249. Du C-J, Sun D-W (2005) Correlating shrinkage with yield, water content and texture of pork ham by computer vision. Journal of Food Process Engineering, 28 (3), 219–232.

References 155

Du C-J, Sun D-W (2006a) Estimating the surface area and volume of ellipsoidal ham using computer vision. Journal of Food Engineering, 73 (3), 260–268. Du C-J, Sun D-W (2006b) Automatic measurement of pores and porosity in pork ham and their correlations with processing time, water content and texture. Meat Science, 72 (2), 294–302. Du C-J, Sun D-W (2006c) Correlating image texture features extracted by five different methods with the tenderness of cooked pork ham: a feasibility study. Transactions of the ASAE, 49 (2), 441–448. Farkas BE, Singh RP (1991) Physical properties of air-dried and freeze-dried chicken white meat. Journal of Food Science, 56 (3), 611–615. Girard PJ (1992) Technology of Meat and Meat Products. London: Ellis Horwood. Hermansson AM (1985) Water and fat holding. In Functional Properties of Food Macromolecules (Mitchell JR, Ledward DA, eds). London: Elsevier Applied Science, pp. 273–314. Huang CT, Clayton JT (1990) Relationships between mechanical properties and microstructure of porous foods: Part I. A review. In Engineering and Food. Vol. 1. Physical Properties and Process Control (Spiess WEL, Schubert H, eds). London: Elsevier Applied Science, pp. 352–360. Hullberg A, Johansson L, Lundström K (2005) Effect of tumbling and RN genotype on sensory perception of cured-smoked pork loin. Meat Science, 69 (4), 721–732. Kassama LS, Ngadi MO (2005) Pore structure characterization of deep-fat-fried chicken meat. Journal of Food Engineering, 66 (3), 369–375. Kassama LS, Ngadi MO, Raghavan GSV (2003) Structural and instrumental textural properties of meat patties containing soy protein. International Journal of Food Properties, 6 (3), 519–529. Khalil AH (2000) Quality characteristics of low-fat beef patties formulated with modified corn starch and water. Food Chemistry, 68, 61–68. Kumar VA, Mathew S (2003) A method for estimating the surface area of ellipsoidal food materials. Biosystems Engineering, 85 (1), 1–5. Lawrie RA (1998) Lawrie’s Meat Science. Cambridge: Woodhead Publishing. MacFie HJH, Hedderley D (1993) Current practice in relating sensory perception to instrumental measurements. Food Quality and Preference, 4 (1), 41–49. Martens H and Martens M. (2001). Analysis of two data tables X and Y: Partial Least Squares Regression (PLSR). In Multivariate Analysis of Quality: an Introduction. London: John Wiley & Sons, pp. 111–125. Mathworks (1998) Matlab Reference Guide. Natick: The MathWorks, Inc. Mayor L, Sereno AM (2004) Modelling shrinkage during convective drying of food materials: a review. Journal of Food Engineering, 61 (3), 373–386. McDonald K, Sun D-W (2001a) The formation of pores and their effects in a cooked beef product on the efficiency of vacuum cooling. Journal of Food Engineering, 47 (3), 175–183. McDonald K, Sun D-W (2001b) Pore size distribution and structure of a cooked beef product as affected by vacuum cooling. Journal of Food Process Engineering, 24, 381–403.

156 Quality Measurement of Cooked Meats

McDonald K, Sun D-W, Kenny T (2000) Comparison of the quality of cooked beef products cooled by vacuum cooling and by conventional cooling. Food Science andTechnology – Lebensmittel Wissenschaft und Technologie, 33, 21–29. Meyer F, Beucher S (1990) Morphological segmentation. Journal of Visual Communication and Image Representation, 1 (1), 21–46. Mittal GS, Blaisdell JL (1984) Heat and mass transfer properties of meat emulsion. Food Science and Technology – Lebensmittel Wissenschaft und Technologie, 17, 94–98. Morgan JB, Savell JW, Hale DS, Miller RK, Griffin DB, Cross HR, Shakelford SD (1991) National beef tenderness survey. Journal of Animal Science, 69, 3274–3283. Mulchrone KF, Choudhury KR (2004) Fitting an ellipse to an arbitrary shape: implication for strain analysis. Journal of Structural Geology, 26, 143–153. Ofstad R, Kidman S, Myklebust R, Hermansson AM (1993) Liquid holding capacity and structural changes during heating of fish muscle: cod (Gahus morhua L.) and salmon (Salmo salar). Food Structure, 12, 163–174. Rahman MS (2001) Toward prediction of porosity in foods during drying: a brief review. Drying Technology, 19 (1), 1–13. Rahman MS, Sablani SS (2003) Structural characteristics of freeze-dried abalone – porosimetry and puncture test. Food and Bioproducts Processing, 81 (C4), 309–315. Rahman MS, Perera CO, Chen XD, Driscoll RH, Potluri PL (1996) Density, shrinkage and porosity of calamari mantle meat during air drying in a cabinet dryer as a function of water content. Journal of Food Engineering, 30 (1–2), 135–145. Russ JC (1999) Image Processing Handbook. Boca Raton: CRC Press. Schäfer A, Rosenvold K, Purslow PP, Andersen HJ, Henckel P (2002) Physicological and structural events postmortem of importance for drip loss in pork. Meat Science, 61, 355–366. Thomas JRGB, Finney RL (1984) Calculus and Analytic Geometry. Boston: AddisonWesley Publishing Company. Wang LJ, Sun D-W (2002a) Modelling vacuum cooling process of cooked meat – part 2: mass and heat transfer of cooked meat under vacuum pressure. International Journal of Refrigeration, 25 (7), 862–871. Wang LJ, Sun D-W (2002b) Evaluation of performance of slow air, air blast and water immersion cooling methods in cooked meat industry by finite element method. Journal of Food Engineering, 51, 329–340. Zheng CX, Sun D-W, Zheng LY (2005) Correlating color to moisture content of large cooked beef joints by computer vision. Journal of Food Engineering, 77 (4), 858–863. Zheng CX, Sun D-W, Du C-J (2006a) Estimating shrinkage of large cooked beef joints during air-blast cooling by computer vision. Journal of Food Engineering, 72 (1), 56–62. Zheng CX, Sun D-W, Zheng LY (2006b) Classification of tenderness of large cooked beef joints using wavelet and Gabor textural features. Transactions of the ASAE, 49 (5), 1447–1454. Zorrilla SE, Singh RP (2003) Heat transfer in double-sided cooking of meat patties considering two-dimensional geometry and radial shrinkage. Journal of Food Engineering, 57 (1), 57–65.

Quality Inspection of Poultry Carcasses Bosoon Park US Department of Agriculture, Agricultural Research Service, Richard B. Russell Research Center, Athens, GA 30605, USA

1 Introduction The Food Safety Inspection Service (FSIS) has been mandated to inspect organoleptically each poultry carcass on the line at processing plants in the US. The development of accurate and reliable instruments for on-line detection of unwholesome carcasses – such as cadavers and those that are septicemic, bruised, tumorous, air-sacculitic, and ascetic – is essential to improve the US federal poultry inspection program. Major causes for condemnation of poultry during quality inspection include the following: 1. Cadaver, which is the carcass of a chicken that died from some cause other than slaughter. The skin is reddish because either the animal was already dead at the time of bleeding, or it was not accurately stuck and therefore did not properly bleed out. 2. Septicemia, which is a systemic disease caused by pathogenic microorganisms and/or their toxins in the blood. It may result in a variety of visible changes in the carcass and viscera of an affected bird, including swollen, watery tissues, hemorrhages throughout the animal, and a darkened red to bluish discoloration of the skin. 3. Bruising, which is due to the accumulation of blood in tissues outside the vascular system, resulting in discoloration of some parts of the skin and underlying tissues. 4. A tumor, which is a mass of swollen or enlarged tissue caused by uncontrolled growth of new tissue that has no useful function. 5. Ascites, which is an accumulation of fluid in the peritoneal cavity of the abdomen. 6. Airsacculitis, which is inflammation of the air sacs (membrane lines, air-filled structures) with the accumulation of fluid or exudate within the cavities. Airsacs may be caused by many different organisms (bacteria, mycoplasma, viruses, or fungi). Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


7

158 Quality Inspection of Poultry Carcasses

Poultry products have increased in popularity with US consumers in recent years. The number of poultry slaughtered at federally inspected establishments has increased from 6.9 billion birds in 1993 to 8.9 billion birds in 2004 (USDA, 2005). Previous research showed that a machine vision system could separate wholesome birds from unwholesome birds, including septicemic carcasses and cadaver, with high classification accuracy (Park and Chen, 1994a). Thus, machine vision systems are useful for poultry industry applications, particularly in grading and inspection, because inspection and classification of poultry carcasses is a tedious and repetitive procedure. Daley et al. (1988) reported that machine vision would be feasible for grading poultry production and for identifying parts of poultry carcasses at the processing line. In the mid-1990s, a multispectral imaging system was developed to identify normal, bruised, tumorous, and skin-torn carcasses for the purpose of poultry quality inspection, and to develop a methodology for separating healthy from unwholesome carcasses (Park et al., 1996). From this study, Park and colleagues determined the optimum wavelengths for identifying bruised, tumorous, and skin-torn carcasses; developed software for the processing and analysis of multispectral images in both spatial and frequency domains; and developed a neural network model for classifying unwholesome carcasses. Thus, machine vision with color and spectral imaging can be used successfully for poultry quality inspection. Currently, individual carcasses are inspected by federal inspectors at poultry processing lines, but this visual bird-by-bird inspection is labor-intensive and prone to human error and variability. Development of high-speed and reliable inspection systems to ensure the safe production of poultry during post-harvest processing has become an important issue, as the public is demanding assurance of better and safer food. Machine vision techniques are useful for the agricultural and food industries, particularly in grading and inspection (Sakar and Wolfe, 1985; Miller and Delwiche, 1989; Tao et al., 1990; Precetti and Krutz, 1993; Daley et al., 1994). Machine vision is the technology that provides automated production processes with vision capabilities, which is particularly useful when the majority of inspection tasks are highly repetitive and extremely boring, and their effectiveness depends on the efficiency of the human inspectors. Even though machine vision has evolved into a promising technology for agricultural product applications, among the many factors to be considered in on-line application are processing speed, reliability, and applicability for industrial environments.

2 Poultry quality inspection The inspection and the grading of poultry are two separate programs within the US Department of Agriculture (USDA). Inspection for wholesomeness is mandatory, whereas grading for quality is voluntary. The service is requested by poultry producers and processors. American consumers can be confident that the FSIS ensures that poultry products are safe, wholesome, and correctly labeled and packaged. Under the Federal Meat Inspection Act and the Poultry Products Inspection Act, the FSIS inspects all raw meat

Color imaging for quality inspection 159

and poultry sold in interstate and foreign commerce, including imported products. It also monitors meat and poultry products after they leave federally inspected plants. In addition, the FSIS monitors state inspection programs, which inspect meat and poultry products sold only within the state in which they were produced. The 1968 Wholesome Poultry Products Act requires state inspection programs to be equivalent to the Federal inspection program. If states choose to end their inspection program or cannot maintain this standard, the FSIS must assume responsibility for inspection within that state. In its efforts to protect the safety and integrity of poultry products, the FSIS works with many other agencies within the USDA and other agencies, including state inspection programs, the Food and Drug Administration of the US Department of Health and Human Services, and the Environmental Protection Agency. Since the Federal inspection program began, the poultry industry has grown and changed significantly. In the early 1900s, most meat was slaughtered and used locally; however, nowadays there is a wide variety of meat and poultry products on the market. Meat is slaughtered and processed in sophisticated, high-volume plants, and is often shipped great distances to reach consumers. As the industry has changed, the FSIS has also changed the inspection program. In its early days the primary concern of the inspectors was disease, and they relied almost exclusively on visual inspection of animals, products, and plant operations. Since the mid-1970s, FSIS has been modernizing inspection to reduce costs and make it more scientifically-based. The requirements in the new final rule on Pathogen Reduction and Hazard Analysis and Critical Control Points (HACCP) are designed to minimize the likelihood of harmful bacteria being present in raw meat and poultry products. However, some bacteria might still be present and may become a problem if meat and poultry are not handled properly. The FSIS inspector must have knowledge about the particular species inspected, and the carcasses must fit with the available equipment in the plant. In modern poultry plants, USDA-certified inspectors perform the whole inspection process. Individual, high-speed visual inspection of birds (35 birds per minute) is both laborintensive, and prone to human error and variability. During the past decade, several studies have reported on the developments of automated inspection systems for poultry carcass inspection (Chen and Massie, 1993; Chen et al., 1996a; Park and Chen, 1996).

3 Color imaging for quality inspection 3.1 Detection of splenomegaly Poultry spleen size is an important indicator of whether the poultry should be condemned and must be further examined by human inspectors in processing plants. According to poultry pathologists and veterinarians, if a chicken has an enlarged spleen then the animal is diseased (Schat, 1981; Arp, 1982; Clarke et al., 1990).


Conversely, if a chicken is diseased, the spleen is likely to be enlarged. As a part of the research on the inspection of poultry carcasses for internal diseases, inspecting spleens was suggested as an initial step. This has been added to the further inspections for other disease syndromes such as airsacculitis and inflammatory processes (Domermuth et al., 1978). Inspection of poultry carcasses for their wholesomeness is a complex process. An automated machine vision inspection system must incorporate human knowledge into a computer system with machine intelligence. The vision system development is often a progressive process, with problems conquered one at a time. Substantial progress has been made regarding the machine vision inspection of poultry carcasses (Chen et al., 1998a; Park et al., 1996). An on-line vision system was developed for inspecting tumors, diseases, and skin damage. Using multispectral imaging and fiber-optics, external chicken surfaces were analyzed. The system seemed highly promising for detecting specific poultry disease problems, and was a step forward in the technology of automated poultry inspection. Through the research, imaging techniques were developed for inspecting the internal organs of poultry to identify abnormalities of the spleen. At the same time, the new knowledge developed through this research was contributing to the understanding and development of future advanced technologies in machine vision-based poultry inspection. A spectral imaging method was developed to identify poultry spleen from its surrounding viscera, such as liver and intestine; and an image-processing algorithm that recognizes the spleen in an image and detects splenomegaly (enlargement of the spleen) was developed. As splenomegaly is one indication that processed poultry may not be acceptable for human consumption, because of diseases such as tumors or septicemia, the study explored the possibility of detecting splenomegaly with an imaging system that would assist human inspectors in food safety inspections. Images of internal viscera from 45-day-old commercial turkeys were taken with fluorescent and ultraviolet lighting systems. Image-processing algorithms using linear transformation, morphological filtering, and statistical classification were developed to distinguish the spleen from its background surroundings, and then to detect abnormalities. Experimental results demonstrated that the imaging method could effectively distinguish the spleen from other organs and intestines. The system had 95 percent classification accuracy for the detection of spleen abnormality. The methods indicated the feasibility of using automated machine vision systems to inspect internal organs as an indication of the wholesomeness of poultry carcasses.

3.2 Inspection of the viscera A practical application of food microbiology in poultry processing and marketing might be to ensure clean, wholesome products. However, under commercial production, processing, handling, and marketing conditions, it is not feasible to run microbiological counts (Mountney, 1987) to determine the presence of pathogens on slaughtered birds. For this reason, the current practice of poultry inspection in the processing plant is based on postmortem pathology correlation – i.e. observing signs of abnormalities or diseases from the carcass exterior, body cavity, and viscera. Previous studies (Chen

Color imaging for quality inspection 161

et al., 1998b, 1998c, 1998d; Park et al., 1998a, 1998b) have shown that the systems can separate normal poultry carcasses from abnormal carcasses. The system, however, may not be able perfectly to discriminate individual abnormal carcasses. In addition, procedures that depend only on images of the carcass exterior are insufficient to detect some condemnable conditions, such as airsacculitis and ascites. Therefore, there is a need to acquire additional information, using machine vision, from post-mortem poultry at different locations (such as the body cavity) and/or from different internal organs (including the liver and heart). Color is an important attribute for food inspection (Daley et al., 1994; Tao et al., 1995). With the availability of improved hardware for acquiring color images, and advances in image-processing software (Jang 1993; Nauck and Kruse, 1995), there is now the capability for development of color-vision systems for poultry inspection. Therefore, Chao et al. (1999) have studied color imaging in identifying individual condemnable conditions from poultry viscera. From the study, they determined features for discriminating condemned conditions of poultry viscera and developed the neurofuzzy models for identifying individual poultry viscera condemnations. Poultry viscera of liver and heart were separated into four classes depending on their symptoms, including normal, airsacculitis, cadaver, and septicemia. These images in RGB color space were segmented, and statistical analysis was performed for feature selection. The neuro-fuzzy system utilizes hybrid paradigms of the fuzzy interference system and neural networks to enhance the robustness of the classification processes. The accuracy in separating normal from abnormal livers was between 87 and 92 percent when two classes of validation data were used. For two-class classification of chicken hearts, the accuracy was between 93 and 97 percent. However, when neuro-fuzzy models were employed to separate chicken livers into three classes (normal, airsacculitis, and cadaver), the accuracy was only 83 percent. Combining the features of chicken liver and heart, a generalized neuro-fuzzy model was designed to classify poultry viscera into four classes (normal, airsacculitis, cadaver, and septicemia). In this case, a classification accuracy of 82 percent was obtained.

3.3 Characterizing wholesomeness For poultry quality and safety inspection, scientifically-based innovative inspection technologies are needed that can allow poultry plants to meet government food safety regulations efficiently and also increase competitiveness and profitability to meet consumer demand. Due to successful food safety and quality monitoring applications in other food processing and production agriculture industries, researchers have been developing spectral imaging methods suited to the poultry processing industry. In particular, visible/near-infrared (Vis/NIR) spectroscopic technologies have shown the capability of distinguishing between wholesome and unwholesome poultry carcasses, and detecting fecal contamination on poultry carcasses, by differences in skin and tissue composition. Chen and Massie (1993) used Vis/NIR measurements taken by a photodiode array spectrophotometer to classify wholesome and unwholesome chicken carcasses, and selected wavelengths at 570, 543, 641, and 847 nm based on linear regression for classification.


Using Vis/NIR measurements of fecal contamination of poultry carcasses, Windham et al. (2003a) identified four key wavelengths via principal component analysis at 434, 517, 565, and 628 nm. Through single-term linear regression (STLR), an optimal ratio of 574 nm/588 nm was determined and used to achieve 100 percent detection of contaminates (Windham et al., 2003b). Chao et al. (2003) developed an on-line inspection system to measure the reflectance spectra of poultry carcasses in the visible to near-infrared regions between 431 and 944 nm. The instrument measured the spectra of poultry carcasses at speeds of 140 or 180 birds per minute. TheVis/NIR system can clearly be used to differentiate wholesome and unwholesome poultry carcasses at high speed. These studies include significant findings for the use of spectral reflectance in the visible region, but have not utilized methods of analysis for sample color as perceived through human vision. The International Commission for Illumination (CIE) has established a colorimetry system for identifying and specifying colors, and for defining color standards. Following the establishment of the CIE 1924 luminous efficiency function, the system of colorimetry was developed based on the principles of trichromacy and Grassmann’s laws of additive color mixture (Fairchild, 1998). The concept of colorimetry is that any color can be matched by an additive mixture of three primary colors: red, green, and blue. Because there are three different types of color receptor cones in the eye, all the colors that humans see can be described by coordinates in a three-dimensional color space, which measures the relative stimulations to each type of cone. These coordinates are called tristimulus values, and can be measured in color-matching experiments. The tristimulus values are the amounts of the three primary colors used to achieve a match. A system using broad-band primaries was formalized in 1931 by the CIE. Wavelengthby-wavelength measurement of tristimulus values for the visible spectrum produces the color-matching functions. The tristimulus values for a particular color are labeled (X, Y, Z) in the CIE 1931 system, and are extended such that they can be obtained for any given stimulus, defined by a spectral power distribution (SPD) (Williamson and Cummins, 1983). The SPD can be measured by a spectrophotometer. From the SPD both the luminance and the chromaticity of a color are derived to describe precisely the color in the CIE system. Chao et al. (2005) investigated a quantitative, color-based method suitable for rapid automated on-line sorting of wholesome and unwholesome chickens. They characterized wholesome and unwholesome chicken color in CIE color coordinates. According to their studies, the color-based sensing technique has the potential for rapid automated inspection for wholesomeness of poultry in the visible region. Spectra in the range of 400–867 nm are suitable for poultry carcass inspection on a high-speed kill line using a visible/near-infrared spectrophotometer. CIELUV color was calculated as a simple distance formula and used to classify wholesome and unwholesome poultry carcass samples. They found that the greatest color differences occurred at different combination of wavelengths – at 508 nm and 426 nm; at 560 nm and 426 nm; and at 640 nm and 420 nm. Full-spectrum classification achieved accuracy of 85 percent in identifying wholesome carcasses. Using the 560nm and 426-nm wavelengths, approximately 90 percent classification accuracy was obtained for wholesome carcasses.

Spectral imaging 163

4 Spectral imaging 4.1 Quality characterization A technique for recognizing global or systemic defects on poultry carcasses by using a color-imaging system has been reported. The goals of this study were to process images at speeds of about 180 birds per minute and to use a neural network-based classifier for classification. Color-image-processing procedures involve three steps: background removal, HSI (hue, saturation, intensity) conversion, and histogram calculation. Features of three histograms (hue, saturation, intensity) were used as inputs of the neural network for detecting large-scale defects (e.g. septicemic carcasses, or cadavers). Also, a color-image processing system to detect skin tears, feathers, and bruising was developed by Daley et al. (1994). The HSI could be more useful for poultry carcasses identification than the RGB and XYZ color processing techniques (Daley and Rao, 1990). However, color machine vision for poultry carcass classification was conducted by using a CCD camera which enables the detection of only broadband, visible (400–700-nm) information in the spatial domain. Park and Chen (1994b) developed a multispectral imaging system to detect abnormal poultry carcasses. The machine vision inspection system they developed provides spectral information regarding the object, as well as the spatial information in the visible and near-infrared spectral regions. Using multispectral images, they characterized several different abnormal poultry carcasses, including bruised, tumorous, and skintorn carcasses. From the study, they determined the optimum wavelength for optical filter selection for discriminating such carcasses. 4.1.1 Spectral characterization of poultry carcasses

Multispectral imaging provides image information in the spectral domain as well as in the spatial domain. Specifically, the intensified multispectral imaging system was found to improve sensitivity and to control exposure automatically, and had the capability to calibrate image intensity. The multispectral camera with selected optical filters provided more spectral characteristics of poultry carcasses. The response of the reflectance intensity of each carcass was sensitive to the wavelength of the filter. Based on the six different wavelengths (542, 570, 641, 700, 720, and 847 nm) with 10-nm bandwidth, which were selected by spectrophotometry of the poultry carcasses (Chen and Massie, 1993), the characteristics of the poultry carcasses were distinguishable when interference-filter wavelengths of 542 and 700 nm were installed in the camera. Figure 7.1 shows the spectral response in normal and abnormal carcasses. The reflectance intensity of normal carcasses was not sensitive to the wavelength of the filter. As shown in Figures 7.1a and 7.1b, there was little difference of reflectance intensity between 542- and 700-nm wavelengths. For normal carcass images, the dark area of the body was a shadow of the image. In the case of bruised carcasses, the reflectance intensity with a 542-nm wavelength was much darker than the body intensity when using a 700-nm wavelength (Figures 7.1c and 7.1d). In Figure 7.1c, the dark area on the back was bruised and the right portion of the left leg was skin-torn. Thus,


(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Figure 7.1 Intensified multispectral images of poultry carcasses: (a) normal at 542 nm; (b) normal at 700 nm; (c) bruising at 542 nm; (d) bruising at 700 nm; (e) tumor at 542 nm; (f) tumor at 700 nm; (g) skin-tear at 542 nm; (h) skin-tear at 700 nm.

200

250 200 150 100

50

100 250 200 150 100

50

dy

100

150

50 ody

(a)

0

0

bo

of

0

200

th

bo

of

50

100 Width of b

th

150

ng

200

Le

0

150 Width

50

100

50

of bod

y

0

ng

150

Le

200

dy

ensity Gray-level int

ensity Gray-level int

250

250

0

(b)

Figure 7.2 Gray-level intensity distribution of poultry carcasses scanned with filter wavelength of 542 nm: (a) normal; (b) bruising.

the tissues of the poultry carcasses can be characterized by spectral imaging using different wavelengths. Multispectral imaging had the potential to differentiate tumorous carcasses from normal carcasses. As shown in Figure 7.1e, the dark area at the center of the body was actually a tumor; however, other dark spots were blood clots; thus a wavelength of 542 nm was not effective at distinguishing tumorous carcasses. However, this problem was solved by using a filter of 700 nm – Figure 7.1f clearly shows that the tumorous spectral image at 700 nm was different from that at 542 nm. The combination of these two different wavelengths enabled differentiation of tumorous carcasses. For a skintorn carcass, the reflectance intensity of the muscle was darker than the intensity of the skin when a 542-nm wavelength was used (Figure 7.1g); on the other hand, the reflectance intensity of the muscle (skin-torn area) with a 700-nm wavelength was high (see Figure 7.1h). Thus, the reflectance image intensity provided the capability of differentiating bruised, tumorous, and skin-torn carcasses. The gray-level image intensity of each carcass was compared, to differentiate abnormal carcasses. Figure 7.2 shows the three-dimensional distribution of gray-level image


intensity in the spatial domain. The image intensity of the bruised carcass varied much more than the intensity of the normal carcass. Thus, the variation of reflectance image intensity could be a significant feature in distinguishing between normal and bruised poultry carcasses.

4.2 Detection of skin tumors Currently, each chicken intended for sale to US consumers must by law be inspected post-mortem, by the Food Safety Inspection Service, for wholesomeness (USDA, 1984). Inspectors visually and manually inspect poultry carcasses and viscera on-line at processing plants. The FSIS uses about 2200 poultry inspectors to inspect more than 8 billion poultry per year in 310 poultry slaughter plants nationwide, and this volume is growing. Each inspector is limited to a maximum of 35 birds per minute. Inspectors working at least 8 hours per day in these conditions have a tendency to develop repetitive strain injuries and attention and fatigue problems (OSHA, 1999). Poultry inspection is a complex process. FSIS inspectors are trained to recognize infectious conditions and diseases, dressing defects, fecal and digestive content contamination, and conditions that are related to many other consumer protection concerns. In general, diseases and defects that occur in the processing of poultry can be placed into several categories. There are diseases/defects that are localized in nature, and those that are generalised or systemic (i.e. affect the whole biological system of the bird). Systemic problems include septicemia and toxemia. Studies using visible/NIR spectroscopy (Chen et al., 2000) and reflectance imaging (Park and Chen, 1994b; Chao et al., 2000) have shown good results in inspecting for systemic diseases of poultry; however, localized problems are difficult to detect, and require the use of not only spectral but also spatial information. Examples of localized poultry diseases/defects include skin tumors and inflammatory process. An automated system to inspect for diseases/defects of poultry must be able to measure these attributes and eliminate unwholesome carcasses. Chicken skin tumors are round, ulcerous lesions that are surrounded by a rim of thickened skin and dermis (Calnek et al., 1991). For high-speed inspection a machine vision system is a solution, but advanced sensing capabilities are necessary in order to deal with the variability of a biological product. Multispectral imaging is a good tool in these advanced techniques. Several investigations (Throop and Aneshansley, 1995; Park and Chen, 1996; Park et al., 1996; Wen and Tao, 1998;) have shown that the presence of defects is often more easily detected by imaging at one or more specific wavelengths where the reflectivity of good tissue is notably different from that of damaged tissue. For example, skin tumors in poultry are less reflective in the NIR than good tissue (Park et al., 1996). The measurable indication may be amplified, and therefore more easily detected, when more than one wavelength is imaged and the difference or ratio of the images is measured. Chao et al. (2002a) investigated the selection of wavelengths for a multispectral imaging system to facilitate the analysis of chicken skin tumors, to process and identify features from multispectral images, and to design classifiers for identifying tumors from normal chicken skin tissue. According to their findings, spectral imaging techniques were used to detect chicken skin tumors. Hyperspectral images of tumorous chickens


were taken in the spectral range 420–850 nm. Principal component analysis (PCA) was applied to select useful wavelength bands (465, 575 and 705 nm) from the tumorous chicken images. Then, multispectral image analysis was performed to generate ratioed images, which were divided into regions of interests (ROIs) classified as either tumorous or normal. Image features for each ROI (coefficient of variation, skewness, and kurtosis) were extracted and used as inputs for fuzzy classifiers. The fuzzy classifiers were able to separate normal from tumorous skin with increasing accuracy as more features were used. In particular, use of all three features gave successful detection rates of 91 and 86 percent for normal and tumorous tissue, respectively.

4.3 Detection of systemic disease Regarding machine vision application for poultry quality and safety inspection, several studies have been conducted over recent decades to develop automated poultry inspection systems using multispectral visible/near-infrared (Vis/Nir) imaging algorithms (Swatland, 1989; Chen and Massie, 1993; Liu and Chen 2001; Hsieh et al., 2002; Park et al., 2002; Chao et al., 2003; Liu et al., 2003; Windham et al., 2003a). From these studies key wavelengths were selected from redundant Vis/Nir spectra (Chao et al., 2003), because selection of key wavelengths enabled simplification of data processing methods for accurate detection of defective carcasses. A multi-channel filter corresponding to the selected wavelengths can be implemented within the imaging system. The modern common-aperture camera with multi-channel filters can take multispectral images with a single shot, and this ability is essential to a real-time automatic inspection system (Park et al., 2003). However, key wavelengths may vary from disease to disease, as well as with the poultry’s environment. After selecting the key wavelengths, image-processing algorithms are developed to correct, analyze, and classify the images. With an appropriate imageprocessing procedure, some features can be extracted from multispectral image data to more suitably represent the classification target and thus increase the classification accuracy. Yang et al. (2004) also developed multispectral image-processing algorithms for differentiating wholesome carcasses from systemically diseased ones, specifically those that are septicemic. The multispectral imaging system included a common-aperture camera and a spectrometer with four-channel filters in the visible wavelength range. An image-processing algorithm defined the ROI for accurate differentiation. According to their study, a multispectral imaging system can successfully differentiate wholesome and septicemic carcasses automatically. From Vis/Nir reflectance spectra of poultry carcasses, average CIELAB L∗ (lightness), a∗ (redness), and b∗ (yellowness) values were analyzed. The difference of lightness between wholesome and septicemic carcasses was significant. The multispectral imaging system included four narrow-band interference filters for 488-, 540-, 580-, and 610-nm wavelengths. The 16-bit multispectral images of poultry carcasses were collected for image processing and analysis. Image-processing algorithms, including image registration, flat-field correction, image segmentation, region of interest identification, feature measurement, and symptom recognition, were developed to differentiate septicemic from wholesome carcasses.


For the image processing, a 610-nm wavelength was used to create a mask to extract chicken images from the background. The average reflectance intensities at 488, 540, 580, and 610 nm from different parts of the fron of the carcass were calculated. Moreover, four normalization and differentiation methods between two wavelengths were also calculated for comparison. Decision trees were applied for generating thresholds for differentiating septicemic carcasses for wholesome ones. The results showed that, using an average intensity of 580 nm in the region of interest, 98 percent of septicemic carcasses and 96 percent of wholesome carcasses were efficiently identified.

4.4 Detection of heart disease Visual inspection of poultry viscera is one of the tasks currently performed by human inspectors at poultry slaughter plants searching for discrepancies resulting from diseases. Because of the significance of poultry viscera in the poultry inspection process, full automation of poultry inspection requires the development of techniques that can effectively identify individually contaminated conditions of poultry viscera. Studies on the development of methods for automated inspection of poultry viscera have focused on morphological measurements of internal organs. Using UV light to segregate the spleen from other internal organs, Tao et al. (1998) used spleen enlargement measurements to classify wholesome and unwholesome poultry carcasses. In classifying poultry diseases from liver and heart images, Chao et al. (1999) reported that RGB color information could be effectively used for differentiating normal livers from airsacculitis and cadaver livers. However, the RGB color images of chicken hearts could not be effectively used for the separation of diseased poultry carcasses. Instead, using narrow band (rather than broadband RGB), images of chicken hearts were effective for the separation of systemically diseased poultry carcasses. Highresolution images, rather than simple monochromatic data, were gathered to give more flexibility in applications – such as generating size and morphological information, or detecting more localized conditions. Spectral imaging measures the intensity of diffusely reflected light from a surface at one or more wavelengths with narrow band-passes. The resulting data for each carcass are three-dimensional (two spatial dimensions and one spectral dimension). Because of the potentially large size of these data sets, spectral imaging often involves three steps: measuring the spectra of whole samples at many wavelengths, selection of optimal wavelengths, and collection of images at selected wavelengths (Muir, 1993; Favier et al., 1998). In general, a Vis/Nir spectrophotometer is chosen to measure the spectra because of its previous success in providing useful information about chicken carcasses (Chen et al., 1996b). From a set of relatively contiguous spectra, it is possible to characterize spectral features with a potential to differentiate diseases. Several methods of wavelength selection have been reported (Saputra et al., 1992; Chen and Massie, 1993). These include combination of spectra, prior knowledge of spectral characteristics, and mathematical selection based on the spectral difference or statistical correlation of the reflection with diseased status. Chao et al. (2001) utilized discriminant analysis on a subset of the available wavelengths.


A multispectral image acquisition system could be implemented in several ways – by using a filter wheel, a liquid-crystal tunable filter (LCTF), an acousto-optics tunable filter (AOTF) several cameras with different filters, and a single camera with a beamsplitter. A critical issue that should be considered in real-time (at least 35 birds per minute, which equates to the speed of a human inspector) operation of these devices is the amount of time between sequentially acquired images at different wavelengths. This is a function of both the image acquisition speed and the switching band speed. Electromechanical filter wheels are limited in the speed of switching filters. Improvement in LCTF technology enables a LCTF system superior to electromechanical filter wheels in both speed and flexibility of spectral selection (Evans et al., 1997). The time required for the LCTF to switch into the next wavelength is approximately 50 ms (Mao and Heitschmidt, 1998). However, this still makes the system unsuitable for synchronization with moving objects, which is necessary for high-speed inspection. Recent advances in optical design make the four-band imager, based on stationary filters and a beamsplitter, a promising technique for real-time operation. It has the advantage of no moving parts and the simultaneous capture of images at four different wavelengths with good image registration. Using this system, Chao et al. (2001) investigated optical spectral reflectance and multi-spectral image-analysis techniques to characterize chicken hearts for real-time disease detection. Spectral signatures of five categories of chicken hearts (airsacculitis, ascites, normal, cadaver, and septicemia) were obtained from optical reflectance measurements taken with a Vis/Nir spectroscopic system in the range of 473–974 nm. Multivariate statistical analysis was applied to select the most significant wavelengths from the chicken-heart reflectance spectra. By optimizing the selection of key wavelengths for different poultry diseases, four wavelengths were selected (495, 535, 585,

Normal

Airsacculitis

Ascites

Cadaver

Septicemia

Figure 7.3 Detection of poultry systemic disease using multispectral heart images at 495, 535, 585, and 605 nm.


and 605 nm). Figure 7.3 shows the detection of poultry systemic disease using multispectral heart images at 495, 535, 585, and 605 nm. The multispectral imaging system utilizes four narrow-band filters to provide four spectrally discrete images on a single CCD focal plane. Using the filters at the wavelengths selected from the reflectance spectra, it was possible easily to implement multispectral arithmetic operations for disease detection. Based on statistical analysis of spectral image data, the multispectral imaging method could potentially differentiate individual diseases in chicken hearts in real-time. All categories except cadaver were separable with accuracy greater than 92 percent by discrimination algorithms involving differences of average image intensities.

4.5 Identification of systemic disease According to the Food Safety and Inspection Service (FSIS) of the USDA, performance standards are set at zero tolerance for two Food Safety categories (i.e. fecal contamination, and infectious condition such as septicemia and toxemia). For poultry plants to meet federal food safety regulations and satisfy consumer demand while maintaining their competitiveness, the FSIS has recognized the need for new inspection technologies (USDA, 1985), such as automated machine-vision based inspection systems. Recent research has investigated the development of automated poultry inspection techniques based on spectral imaging. Chao et al. (2002) developed a multispectral imaging system using 540- and 700-nm wavelengths, and obtained accuracies of 94 percent for wholesome and 87 percent for unwholesome poultry carcasses. With hyperspectral imaging, Park et al. (2002) achieved 97–100 percent accuracy in identifying fecal and ingesta contamination on the surface of poultry carcasses using images at the 434-, 517-, 565-, and 628-nm wavelengths. They found that spectral images present spectral and spatial information from the surface of broiler carcasses, which is essential for efficient identification of contaminated and systemically diseased broilers. Not only can multispectral imaging achieve high classification accuracies, this non-destructive method also shows potential for on-line inspections at high-speed processing plants. Based onVis/Nir spectroscopic analysis (Hruschka, 1987), previous studies have also shown that key wavelengths are particularly useful for the identification of diseased, contaminated, or defective poultry carcasses (Chao et al., 2003; Windham et al., 2003a). After selection of key wavelengths, filters corresponding to those wavelengths can be implemented for multispectral image acquisition. Image-processing algorithms are then developed to enhance and analyze the images. With appropriate image-processing procedures, some features can be extracted from multispectral images to more suitably represent the classification target and increase the classification accuracy. Yang et al. (2006) have developed a simple method for differentiating wholesome carcasses from systemically diseased carcasses using signatures of Vis/Nir multispectral images. Image-processing algorithms extract image features that can be used to determine thresholds for identifying systemically diseased chickens. According to their study, color differences between wholesome and systemically diseased chickens can be used to select interference filters at 488, 540, 580, and 610 nm for the multispectral imaging system. An image-processing algorithm to locate the


region of interest was developed in order to define four classification areas on each image, including whole carcass, region of interest, upper region, and lower region. Three feature types – average intensity, average normalization, and average difference normalization – were defined using several wavelengths for a total of 12 classification features. A decision-tree algorithm was used to determine threshold values for each of the 12 classification features in each of the 4 classification areas. The feature “average intensity” can be used to identify wholesome and systemically diseased chickens better than other features. Classification by average intensity in the region of interest using 540- and 580-nm wavelengths resulted in the accuracies of 96 and 97 percent for the classification of wholesome and systemically diseased chickens at 540 nm, respectively. This simple differentiation method shows potential for automated on-line chicken inspection.

4.6 Quality inspection by dual-band spectral imaging Over the past three decades, poultry production has greatly increased and the processing speed at slaughter plants has tripled (USDA, 1996a). Due to the massive production of poultry and the inherent variability and complexity in individual birds, there are great challenges for further improvement of the existing organoleptic inspection methods. To design an effective machine vision system for on-line applications, vision hardware functionality needs to be considered during the development of software (Park et al., 1995). A spectral imaging system measures the intensity of diffusely reflected light from a surface at several wavelengths. The reflected light contains information regarding the area close to the skin surface of broiler carcasses. Using intensities at six different spectral wavelengths (540, 570, 641, 700, 720, and 847 nm), several spectral image algorithms to differentiate wholesome carcasses from unwholesome carcasses have been developed (Park and Chen 1996; Park et al., 1996). In this case, comparison of images at two or more wavelengths provides robustness for classifying spectral images. Since the process of analyzing a digital image to identify certain objects is inherently computationally intensive, it is advantageous optically to pre-process the image, extracting only those wavelengths which provide useful information. A pilotscale facility has been constructed specifically for developing the machine-vision based systems for on-line poultry inspection. The facility has been utilized for evaluating individual vision components and testing the workability of spectral imaging algorithms (Park and Chen, 1998). Chao et al. (2000) designed a real-time machine vision system, including vision hardware and software components integration, which can be adapted to on-line processing at poultry slaughter plants. Object-oriented analysis was employed to identify the system’s responsibility for individual components. A real-time machine vision inspection system was implemented in the pilot-scale facility. The system’s performance was optimized for on-line classification of normal and abnormal poultry carcasses. According to their studies, two sets of dual-camera systems were applicable for on-line inspection of poultry carcasses: one to image the front of the bird and the other to image the back. Each system consisted of two identical CCD cameras equipped with interference

Poultry image classifications 171

filters of 540 nm and 700 nm with 10-nm bandwidth. The first set of dual-cameras captured the spectral images simultaneously, followed by the second set of dual-cameras. Object-oriented analysis was performed to identify the attributes of individual software components and the relationships between these software components. These individual software components were then organized by the object patterns to form a software architectural framework for on-line image capture, off-line development of classification models, and on-line classification of carcasses into wholesome and unwholesome categories. For the model development, the accuracies to differentiate between wholesome and unwholesome carcasses were 96 and 88 percent at 540 and 700 nm, respectively, for the front images; and 95 and 85 percent at 540 and 700 nm, respectively, for the back images. According to the on-line classification using neural network models, the imaging system used for scanning the fronts of carcasses performed well, with accuracies of 91, 98 and 95 percent for normal, abnormal, and combined carcasses, respectively. However, the system accuracy tested from the back images produced accuracies of 84, 100 and 92 percent for normal, abnormal, and combined carcasses. Thus, dual-camera based spectral imaging system with selective wavelength filters can be effectively used for on-line poultry quality inspection.

5 Poultry image classifications 5.1 Airsac classification by learning vector quantization Since it was recognized that computer imaging would greatly improve the inspection procedures, much work has been devoted to automatic inspection for wholesomeness in chicken carcasses. Most of the research is based on different optical techniques, mainly spectroscopy for the classification of wholesome, septicemic, and cadaver carcasses. Chen and Hruschka (1998) performed on-line trials of a system for chicken carcass external inspection, based on Vis/NIR reflectance. The system was able successfully to identify 95 percent of the carcasses at a speed of 70 birds per minutes. Fiber-optic spectroscopy was also used for the classification of diseases in slaughtered poultry carcasses (Park et al., 1998a). Park et al. (1998b) also proposed the combination of multispectral imaging and neural network classification models. In that research, two cameras with interference filters at 540 nm and 700 nm and a back-propagation neural network algorithm were used for the inspection of wholesomeness in poultry carcasses. As for the detection of lesions commonly observed in the body cavity, Chao et al. (1998) analyzed the size and coloration of liver in infected poultry. In related research (Tao et al., 2000), the size and color features of infected enlarged spleen in turkeys were studied. Both studies were performed under laboratory conditions, with the viscera prepared prior to the experiments. Color processing is also very competent for identifying agricultural problems. Ibarra et al. (2002) developed a method for the classification of airsacculitis lesions in chicken carcasses induced by secondary infection with Escherichia coli. They established a procedure for controlled induction of airsacculitis as well as RGB color transformation for optimal classification. In addition, neural network classification was implemented


for color features from airsacculitis, using the learning vector quantization (LVQ) technique. According to their research, the variation in color features observed during the evolution of airsacculitis in chicken carcasses can be exploited to classify the disease using digital imaging and neural networks. For the supervised classification, a knowledge base set of normalized RGB values (corresponding to negative, mild, and severely infected airsacs images) were obtained. Statistical data exploration indicated no significant difference between the color features of mild and severely infected airsacs; a significant difference, however, was found between infected and negative tissues. A neural network using the learning vector quantization algorithm classified the data from infected and negative categories. Resubstitution and hold-out errors were calculated, giving an overall classification accuracy of 96 percent. The method developed in this research has potential for integration into a computer-assisted inspection system for wholesomeness at the poultry processing plants.

5.2 Quality classification by texture analysis The features to be extracted from intensity information were mean, variance, and histogram of intensity. Even though the reflectance intensity measurement of the spectral images provided useful information in the spatial domain for differentiating poultry carcasses, these features were too sensitive to the variation in light intensity and spatial dependency. Textural analysis methods, specifically Fourier power spectra analysis and fractal analysis in the frequency domain, on the other hand, only depend on the spectral frequency distribution on the image surface. This textural information is invariant to the variation of light intensity and spectral dependency, rather than spatial dependency. Texture is the term used to characterize the tonal or gray-level variation in an image. Texture is an important discriminating surface characteristic which can aid in segmentation and classification of the region. Regions in an image cannot be classified until the image has been segmented, but segmentation requires knowledge of region boundaries. Hence, most methods of texture analysis operate on sub-images when the composition of the image is unknown. This leads to a compromise between classification accuracy and resolution. A smaller sub-image would not be a good representative, while a larger sub-image would result in poor segmentation resolution. Therefore, the sub-images need to be selected based on the consideration of carcass image size. Fourier power spectrum analysis and fractal analysis were introduced for multispectral spectral image classification of poultry carcasses. 5.2.1 Spectral poultry image classification in the frequency domain

For fast Fourier transform (FFT) analyses, all images were transformed by equation (7.1): mu nv 1 + f (m, n)exp −j2π MN m=0 n=0 M N M−1 N−1

F (u, v) =

(7.1)

To increase computational speed, the FFT algorithm was used. The input image was recursively reordered to the form suitable for FFT calculation. Each spectral component


was calculated by using factor numbered look-up tables to optimize speed at the expense of memory requirement. Since many image frequency spectra decrease rapidly with increasing frequency, their high-frequency terms have a tendency to become obscured when displayed in the frequency domain. Therefore, the equation below was used for Fourier power spectrum representation instead of |F(u, v)|: D(u, v) = 50 ∗ log(1 + |F (u, v)|)

(7.2)

Also, to display the full size of the Fourier power spectrum, the origin of the image in the frequency domain was shifted to the coordinate of (N/2, N/2). Since only the Fourier spectrum of the image was preserved, it was impossible to use invert FFT (IFFT) to get back to the original image. Therefore, users should save it to a different file if they wish to retain the original image. The radial distributions of values in |F|2 are sensitive to textural coarseness. A coarse texture will have high values of |F|2 concentrated near the origin, while a smoother texture will have more spread (i.e. like a ring). Similarly, angular distributions of the values of |F|2 are sensitive to the directionality of the texture. Thus, a directional texture will have high values concentrated around the perpendicular lines (like wedges). 5.2.2 Fast power spectra of spectral images The Fourier power spectra provide the coarseness of the texture of spectral images. The 128 × 128 (16 384 pixels) image was cropped out of the whole body to generate the power spectrum (Park et al., 1996a). Figure 7.4 shows the region of interest in wholesome and unwholesome (bruised, skin-torn, and tumorous) carcass images and corresponding FFT at different wavelengths of 542 and 700 nm. The Fourier spectrum of wholesome carcasses is distinguishable from that of unwholesome carcasses. As shown in each spectrum, there was little difference in the power spectrum of the spectral image between 542 nm and 700 nm, except in the skin-torn carcass image. For normal carcasses, the power spectrum was spread around the x-axis and concentrated around horizontal lines. Thus, the textural feature of normal carcasses in the frequency domain had a more directional distribution. On the other hand, the power spectra of bruised, tumorous, and skin-torn carcasses concentrated near the origin. The features in the frequency domain provided the texture coarseness. Since the radial distributions of values in the Fourier power spectrum were sensitive to the texture coarseness of the image in the spatial domain, a coarse texture had the high values of the power spectrum concentrated near the origin, while smoother textures had more spread. Therefore, the Fourier power spectrum was useful to differentiate normal carcasses from abnormal carcasses (bruised, tumorous, and skin-torn carcasses) because it provides spectral information and the features in the frequency domain are spatial-independent. 5.2.3 Fractal analysis

“Fractal” is a term used to describe the shape and appearance of objects which have the properties of self-similarity and scale-invariance. The fractal dimension is a scaleindependent measure of the degree of boundary irregularity or surface roughness (Park et al., 1996a).


(a)

(b)

(c)

(d)

(e)

(f )

(g)

(h)

Figure 7.4 Region of interest of poultry carcass images (128 × 128 pixels) and corresponding FFT at different wavelengths: (a) normal at 542 nm; (b) normal at 700 nm; (c) bruising at 542 nm; (d) bruising at 700 nm; (e) skin-tear at 542 nm; (f) skin-tear at 700 nm; (g) tumor at 542 nm; (h) tumor at 700 nm.

Assume that the intensity I of a square image of size N × N is given by I = I(x, y) where 0 ≤ x, y < N − 1 and a displacement vector is defined as w = (x, y), where x and y are integers. The integer restriction on x and y results from the discrete nature of the image storage system. Minimum non-zero displacements are thus one picture element horizontally or vertically. Finally, the difference of the image intensity at a point (x, y) for a specific displacement vector w is defined by the following equation: Iw (x, y) = I(x, y) − I(x + x, y + y)

(7.3)


Table 7.1 Fractal features of poultry carcasses in the frequency domain. Wavelength (nm) 542

Normal Tumorous Bruised Skin-torn

700

Fractal dimension

Roughness

Slope

Fractal dimension

Roughness

Slope

2.3547 2.3611 2.4890 2.4900

0.6453 0.6367 0.5110 0.5100

−2.2906 −2.2680 −2.0220 −2.0196

2.3640 2.4015 2.3756 2.4750

0.6356 0.5989 0.6240 0.5246

−2.2710 −2.1970 −2.2490 −2.0490

The above equation gives the difference of the image intensity of a picture along with a specific displacement vector w, whose beginning is at a point (x, y) and whose end is at a point (x + x, y + y). For example, if w = (1, 0) then for point (x, y) we can construct the difference of the image intensities simply by calculating I(x, y) − I(x + 1, y) for all combinations of x and y. In practice, the maximum value of x or y would be limited to N − 2 to remain within the boundaries of the image. The Image Processing Tools for Windows in-house software ARS scientists developed first finds the FFT of the image on the active window, then, from the result of the FFT, the fractal of the images were calculated. The results of fractal analyses displayed on the window were saved to the file name FRACTAL.DAT and POWER.DAT. The fractal dimension D and roughness parameter H were calculated by: −Slope = 1 + 2H = 7 − 2D

(7.4)

Roughness parameter H ranges from 0 to 1. When H is close to 0, the surface is the roughest. When the value of H is close to 1, the surface is relatively smooth. From these results, the roughness surface of an image can be quantized. Fractal dimension, roughness, and slope of intensity changes were calculated from the Fourier spectra of each carcass. Table 7.1 shows the fractal values of normal, tumorous, bruised, and skin-torn carcasses at the wavelengths of 542 and 700 nm. Based on the spectral images scanned by the 542-nm wavelength, the fractal dimension of normal carcasses was smaller than that of abnormal carcasses. However, the roughness and slope of the normal carcass were larger than those fractal features of the tumorous, bruised, and skin-torn carcasses. The fractal dimension of the bruised carcasses was much the same as that of skin-torn carcasses, which was even larger than the fractal dimension of tumorous carcasses. The roughness and slope values of the bruised carcasses were similar to those values of skin-torn carcasses, but lower than in tumorous carcasses. However, the fractal features of the spectral images scanned by the 700-nm wavelength were not consistent compared with the results of the 542-nm wavelength – i.e., the fractal dimension of bruised carcasses was lower than that of tumorous carcasses, and the roughness and the slope values of the bruised carcasses were higher than those of tumorous carcasses. Thus, the fractal features of the poultry carcasses varied with the wavelength of spectral images. Finally, the fractal dimension of the


normal carcasses was lower and the roughness and the slope of the normal carcasses were higher than in abnormal carcasses in the spectral images of 700-nm wavelength. 5.2.4 Neural network models A feed-forward backpropagation neural network algorithm was used for classifying poultry carcasses. Because of prediction-related problems, the feed-forward network structure was suitable for handling non-linear relationships between input and output variables. Backpropagation was most frequently used for feed-forward networks. The mathematical description of the backpropagation to be used for classification was reported (Park et al., 1994). The network has an input layer with 256 input nodes, an output layer with 2 output nodes, and a hidden layer with 6 hidden nodes. Each layer was fully connected to the succeeding layer. During learning, information was also propagated back through the network and used to update the connection weights. The aim of the learning process is to minimize the global error of the system by modifying the weights. Given the current set of weights, it is necessary to determine how to increase or decrease them in order to decrease the global error. For the backpropagation algorithm, it is important to set an appropriate learning rate. Changing the weights as a linear function of the partial derivative of the global error makes the assumption that the error is locally linear which is defined by the learning coefficient. To avoid divergent behavior of the network model, it is important to keep the learning coefficient low. However, a small learning coefficient can lead to very slow learning. The “momentum” was implemented to resolve this dichotomy. The delta weight equation was modified so that a portion of the previous delta weight was fed through to the current delta weight. The momentum term allows a low learning coefficient but fast learning.

5.2.4.1 Spectral poultry image data for neural network models The region of interests (ROI) of the image to be analyzed was 128 × 128 (= 16 384) pixels; however, because of the limitation of the number of the neural network input nodes, the size of ROI was reduced to 16 × 16 (= 256) pixels, which was used for the neural network models as input data. These input data were generated by averaging 8 × 8 image pixels of each chicken gray-intensity image. Figure 7.5 shows the image data generated in the spatial domain and spectral domain for neural network models.

(a)

(b)

(c)

(d)

Figure 7.5 Multispectral images (16 × 16 pixels) at 542-nm wavelength, for neural network model: (a) gray intensity of tumorous carcass; (b) FFT of tumorous carcass; (c) gray intensity of normal carcass; (d) FFT of normal carcass.


5.2.4.2 Neural network pattern classification The neural network (NN) classifiers were developed and validated to differentiate tumorous carcasses from normal carcasses based on the image data generated by the Neural Network Image Date Generation Tool included in in-house software. The NN model had 256 input nodes, a hidden layer with 16 hidden nodes, and 2 outputs. Based on the testing results, using a total of 216 carcass images including 108 normal and 108 tumorous, the classification accuracy of neural network models for separating tumorous carcasses from normal ones was 91 percent. When two spectral images (542- and 700-nm wavelengths) were combined and used as input data for the NN model to reduce the variability of intensity distribution (considering the position of the tumor on the body) in the spatial domain, the classification model performed perfectly. None of the tumorous carcasses were classified as normal carcasses. Thus, the combined information gained from different spectral images improved the performance of neural network models in classifying tumorous from normal carcasses.

5.3 Supervised algorithms for hyperspectral image classification In addition to infectious conditions of poultry carcasses, the FSIS is also concerned about fecal contamination; in their food safety performance standards, there is zero tolerance (USDA, 1996b). In order to select the optimum classifier for identifying surface contaminants of poultry carcasses, the performance of six different supervised classification algorithms were investigated and compared. A pushbroom line-scan hyperspectral imager was used for hyperspectral image acquisition with 512 narrow bands between 400- and 900-nm in wavelength. Feces from three different parts of the digestive tract (duodenum, ceca, colon) and ingesta were considered as contaminants. These contaminants were collected from broiler carcasses fed with corn, milo, and wheat with soybean mixture. 5.3.1 Hyperspectral imaging system

A hyperspectral imaging system (Park et al., 2002) was used to collect spectral images of contaminated and uncontaminated poultry carcasses. A transportable imaging cart was designed to provide both portability and flexibility in positioning both the lights and the camera system. The cart also contained a computer, power supplies, and other equipment for hypercube date collection. Lighting requirements were evaluated and adjusted for quality image acquisition. The imaging system consisted of an imaging spectrograph with a 25-µm slit width and an effective slit length of 8.8 mm – Grating Type I (ImSpector V9, PixelVision, Beaverton, Oregon); a high resolution CCD camera (SensiCam Model 370KL, Cooke Corporation, Auburn Hills, MI); a 1.4/23-mm compact C-mount focusing lens (Xenoplan, Schneider, Hauppauge, NY) and associated optical hardware; motor for lens motion control (Model RSP-2T, Newport Corp., Irvine, CA); a frame-grabber (12-bit PCI interface board, Cooke Co, Auburn Hills, MI); and a computer (Pentium II, 500 MHz). The prism-grating-prism spectrograph had a nominal spectral range of 430–900 nm with a 6.6-mm axis, and attached to the camera


for generating line-scan images. The spectrograph had a nominal spectral resolution of 2.5 nm, and was connected to a 2/3-inch silicon-based CCD sensor with 1280 × 1024 pixel resolution. The camera was thermoelectrically cooled and had a spectral response from 290 to 1000 nm with a maximum readout time of 8 fps. For consistent illumination of poultry carcasses, the lighting system consisted of the 150-watt quartz halogen DC stabilized fiber-optic illuminator (Fiber-Lite A240, Dolan-Jenner, Inc., Lawrence, MA), a lamp assembly, fiber-optic cables, and 10-inch illuminating size of quartz halogen line lights (QF5048, Dolan-Jenner, Inc., Lawrence, MA). 5.3.2 Classification methods

Six supervised classification methods were examined in this study for selecting optimum classifiers to identify contaminants on the surface of broiler carcasses: parallelepiped, minimum distance, Mahalanobis distance, maximum likelihood, spectral angle mapper, and binary encoding classifier. Parallelepiped classification uses a simple decision rule to classify hyperspectral data. The decision boundaries form an n-dimensional parallelepiped in the image data space. The dimensions of the parallelepiped are defined based upon a standard deviation threshold from the mean of each selected class. If a pixel value lies above the low threshold and below the high threshold for all n bands being classified, it is assigned to that class. The minimum distance method uses the mean vectors of each endmember and calculates the Euclidean distance from each unknown pixel to the mean vector for each class. All pixels are classified to the nearest class unless a standard deviation or distance threshold is specified, in which case some pixels may be unclassified if they do not meet the selected criteria. Maximum likelihood classification assumes that the statistics for each class in each band are normally distributed, and calculates the probability that a given pixel belongs to a specific class. Unless a probability threshold is selected, all pixels are classified. Each pixel is assigned to the class that has the highest probability. The Mahalanobis distance classification is a direction-sensitive distance classifier that uses statistics for each class. It is similar to the maximum likelihood classification, but it assumes that all class co-variances are equal and therefore processing time is faster. All pixels are classified to the closest region of interest (ROI) class unless a distance threshold is specified, in which case some pixels may be unclassified if they do not meet the threshold. For more details about classification algorithms, readers are referred to Richards and Jia (1999). The spectral angle mapper (SAM) is a physically-based spectral classification that uses an n-dimensional angle to match pixels to reference spectra. The algorithm determines the spectral similarity between two spectra by calculating the angle between the spectra, treating them as vectors in a space with dimensionality equal to the number of bands. SAM compares the angle between the endmember spectrum vector and each pixel vector in n-dimension space. Smaller angles represent closer matches to the reference spectrum. More details are presented in Kurse et al. (1993). The binary encoding classification method encodes the data and endmember spectra into 0s and 1s based on whether a band falls below or above the spectrum mean. An exclusive OR function is used to compare each encoded reference spectrum with the encoded data spectra and a classification image produced. For more details about binary encoding classification algorithm, see Mazer et al. (1988).


After all supervised classification methods had been applied to the hyperspectral ROI data, the post-classification method (a confusion matrix in this case) was applied for the optimum selection of the classification method to identify fecal and ingesta contaminants. For the assessment of classification accuracy, a confusion matrix was analyzed to determine the accuracy of a classification results by comparing a classification result with ground truth ROI information. The kappa coefficient was also calculated to compare the accuracy of different classifiers. The kappa coefficient is an indicator or overall agreement of a matrix and accounts for all the elements in a confusion matrix. The kappa coefficient (κ) can be obtained by: N χkk − χk χk κ=

k

N2

−

k

(7.5)

χk χk

k

where N = total number of pixels in all ground truth classes, χkk = sum of confusion matrix diagonals, χk = sum of ground truth pixels in a class, and χk = sum of classified pixels in that class. The kappa coefficient is always less than or equal to 1. A value of 1 implies perfect agreement, and values less than 1 imply less than perfect agreement. 5.3.3 Hyperspectral image characteristics for classification

In order to select the optimum classification method for fecal and ingesta contaminant identification on poultry broiler carcasses, six different supervised classification methods were investigated and the results were compared. Figure 7.6 shows a typical hyperspectral image of uncontaminated (Figure 7.6a) and surface contaminated ROIs (Figure 7.6b). In this sample, 25 pixels were observed as being duodenum, 27 pixels as ceca, 78 pixels as colon, 93 pixels as ingesta, and 195 pixels as skin. Actually, the skin included breast, thigh, and wing for classification.

Duodenum

Ceca Colon

Ingesta (a)

(b)

Figure 7.6 ROI of a corn-fed poultry carcass: (a) clean (uncontaminated); (b) fecal contaminant. ROI: duodenum (25 pixels), ceca (27 pixels), colon (78 pixels), ingesta (93 pixels), skin (195 pixels).


80 Duodenum Ceca Colon Ingesta Skin (thigh) Skin (breast) Skin (wing)

70

Reflectance, percent

60 50 40 30 20 10 0 400

450

500

550

600

650

700

750

800

850

900

Wavelength, nm Figure 7.7 Mean spectra of fecal and ingesta contaminant ROIs from corn-fed poultry broiler carcass.

Figure 7.7 is the corresponding spectrum for each ROI. Each spectrum indicated duodenum, cecal, colon, ingesta, thigh, breast, and wing, respectively. Typically, the spectra from contaminants gradually increased with wavelength from 420 to 730 nm, whereas the reflectance spectra of skin increased to about 520 nm but decreased and then increased again from about 550 nm upwards. The reflectance spectra of skin were much higher than those of the contaminants. 5.3.4 Comparison of classification methods

Figure 7.8 shows six different classification maps that allow visualization of results of each classification method tested to identify fecal and ingesta contaminants on surface of broiler carcasses. The parallelepiped classifier identified duodenum, ceca, and colon with high accuracy. However, many ingesta pixels were misclassified as duodenum (Figure 7.8a). Most duodenum, cecal, and colon contaminants, except ingesta, were also classified correctly by minimum distance classifier (Figure 7.8b). The Mahalanobis distance classifier also classified fecal contaminants with high accuracy, yet most ingesta contaminants were misclassified as duodenum and uncontaminated skin surfaces were also misclassified as duodenum (false positive) (Figure 7.8c). The results of the maximum likelihood classifier were similar to those of the Mahalanobis distance classifier. The duodenum, cecal, and colon contaminants were classified with a minimal misclassification rate. The misclassification of ingesta was much lower than with the Mahalanobis distance classifier; however, many false positive pixels for uncontaminated skin were found (Figure 7.8d). The spectral angle mapper classifier also identified most fecal and ingesta contaminants with high classification accuracy. However, with this classifier many pixels on the skin and vent area were misclassified as duodenum (Figure 7.8e). Even though the classification accuracy was not high


(a)

(b)

(c)

(d)

(e)

(f )

Figure 7.8 Classification maps from mean spectra of surface contaminant ROI from corn-fed poultry carcasses: (a) parallelepiped classifier; (b) minimum distance classifier; (c) Mahalanobis distance classifier; (d) maximum likelihood classifier; (e) spectral angle mapper classifier; (f) binary coding classifier. Each color map represents duodenum (first row of from top), ceca (second row), colon (third row), ingesta (fourth row), skin (white), and unclassified or background (black).

enough, the binary coding classifier classified most fecal contaminants and ingesta as well. For this classifier, many pixels on skin were misclassified as colon contaminants (Figure 7.8f). 5.3.5 Accuracy of classifiers for contaminant identification

Six different supervised classification methods were applied for the broiler carcasses fed with three different feeds to compare the accuracy of classification methods for selecting a robust classifier regardless of the diet fed to the poultry. Table 7.2 shows the overall mean accuracies of each classification method as applied to differently-fed broiler carcasses. Both the maximum likelihood and spectral angle mapper classifiers performed with higher accuracy than other classifiers for all fecal and ingesta contaminant identification from all the differently-fed broiler carcasses. For the corn-fed carcasses, the classification accuracies ranged from 64.7 (parallelepiped) to 92.3 (spectral angle mapper) percent. The mean accuracy of classifiers for milo-fed carcasses was slightly lower than for corn-fed carcasses, with the accuracy ranging from 62.9 (binary coding) to 88 (maximum likelihood) percent. For wheat-fed carcasses, the highest mean classification accuracy (91.2 percent) was again obtained from the maximum likelihood classifier. Of the six supervised classification methods, the best classifier for classifying fecal and ingesta contaminants was the maximum likelihood method (90.2 percent), followed by the spectral angle mapper method (89.4 percent),


Table 7.2 Mean accuracy of classification methods for classifying feces and ingesta contaminants in three differently-fed (corn, milo, and wheat) broiler carcasses.

Parallelepiped Minimum distance Mahalanobis distance Maximum likelihood Spectral angle mapper Binary coding a

Corn

Milo

Wheat

Average

64.70 (0.590)a 79.73 (0.760) 69.21 (0.634) 91.44 (0.899) 92.27 (0.908) 66.83 (0.607)

66.48 (0.612) 78.75 (0.747) 70.41 (0.649) 88.02 (0.859) 87.34 (0.849) 62.94 (0.563)

66.86 (0.615) 80.41 (0.767) 71.33 (0.659) 91.16 (0.895) 88.65 (0.865) 63.80 (0.574)

66.01 (0.606) 79.63 (0.758) 70.32 (0.647) 90.21 (0.884) 89.42 (0.874) 64.52 (0.581)

kappa coefficient values are given in parentheses.

the minimum distance method (79.6 percent), the Mahalanobis distance method (70.3 percent), the parallelepiped method and binary coding method (64.5 percent). The kappa coefficients in Table 7.2 indicate overall agreement of a matrix, and accounts for all the elements in a confusion matrix, which is used to calculate overall accuracy in the table. A kappa coefficient of 1 reflects perfect agreement between classification and ground truth data. The kappa coefficients confirmed that the optimum classifiers were the SAM classifier (0.908) for corn, and the maximum likelihood for both milo (0.859) and wheat (0.895), which indicated those classifiers had very good agreement in identifying each contaminant from different diets.

6 Conclusions Food safety is an important issue for public health, because reduction in potential health risks to consumers from human pathogens in food is the most important public concern. The Food Safety and Inspection Service (FSIS) in the USDA sets zero tolerance performance standards for two food safety categories, including fecal contamination and infectious condition such as septicemia and toxemia, during poultry processing. Along with global food safety issues, the FSIS is charged with protecting consumers by ensuring safe and wholesome poultry and poultry products. The FSIS is pursuing a broad and long-term scientifically-based strategy to improve the safety of poultry and poultry products to better protect public health. For poultry plants to meet federal food safety regulations and satisfy consumer demand while maintaining their competitiveness, the FSIS has recognized the need for new inspection technologies, such as automated machine-vision based inspection systems. Several different machine vision systems, including color, multi-, and hyper-spectral imaging, have been developed and tested for poultry quality and safety inspection. For high-speed inspection, machine vision is a solution; however, it requires advanced sensing capabilities for the complexity of poultry carcasses. Multispectral imaging is a good tool in these advanced techniques because of its capability to detect both unwholesomeness and contamination using two or more specific wavelengths which reflect the condition of poultry carcasses. Along with selective image processing and analysis software, a multispectral imaging system can be effectively implemented

References 183

real-time, on poultry processing lines, at the speed the industry requires (currently 140 birds per minute). Hyperspectral imaging is also an extremely useful tool to analyze thoroughly the spectra of the surface of poultry carcasses, because hyperspectral image data contain a wide range of spectral and spatial information. A hyperspectral imaging system with simple image-processing algorithms could be effectively used for the detection of both contaminants and infectious disease on the surface of broiler carcasses. Further analyses of hyperspectral imagery enable identification of the type and sources of various contaminants and systemic diseases, which can determine critical control points to improve HACCP for the federal poultry safety program. Because the concerns of today’s inspectors are broader, and include unseen hazards such as microbiological and chemical contamination, hyperspectral imaging techniques will be widely used for poultry quality and safety inspection.

References Arp JH (1982) Pathology of spleen and liver in turkeys inoculated with Escherichia Coli. Avian Pathology, 11, 263–279. Calnek BW, Barnes HJ, Beard CW, Reid WM, Yoder HW (1991) Diseases of Poultry. Iowa State University Press, pp. 386–484. Chao K, Gates RS, Anderson RG (1998) Knowledge-based control systems for single stem rose production – Part I: systems analysis and design. Transactions ASAE, 41 (4), 1153–1161. Chao K, Chen YR, Early H, Park B (1999) Color image classification systems for poultry viscera inspection. Applied Engineering in Agriculture, 15 (4), 363–369. Chao K, Park B, Chen YR, Hruschka WR, Wheaton FW (2000) Design of a dual-camera system for poultry carcasses inspection. Applied Engineering in Agriculture, 16 (5), 581–587. Chao K, Chen YR, Hruschka WR, Park B (2001) Chicken heart disease characterization by multispectral imaging. Applied Engineering in Agriculture, 17 (1), 99–106. Chao K, Mehl PM, Chen YR (2002) Use of hyper- and multi-spectral imaging for detection of chicken skin tumors. Applied Engineering in Agriculture, 18 (1), 113–119. Chao K, Chen YR, Chan DE (2003) Analysis of Vis/NIR spectral variations of wholesome, septicemia, and cadaver chicken samples. Applied Engineering in Agriculture, 19 (4), 453–458. Chao K, Chen YR, Ding F, Chan DE (2005) Characterizing wholesome and unwholesome chickens by CIELUV color difference. Applied Engineering in Agriculture, 21 (4), 653–659. Chen YR, Hruschka WR (1998) On-line trials of a chicken carcass inspection system using visible/near-infrared reflectance. ASAE Paper No. 983047, ASAE, St Joseph, MI, USA. Chen YR, Massie DR (1993) Visible/near-infrared reflectance and interactance spectroscopy for detection of abnormal poultry carcasses. Transactions ASAE, 36 (3), 863–869.


Chen YR, Huffman RW, Park B. (1996a) Changes in the visible/NIR spectra of chicken carcasses in storage. Journal of Food Process Engineering, 19, 121–134. Chen YR, Huffman RW, Park B, Nguyen M (1996b) Transportable spectrophotometer system for on-line classification of poultry carcasses. Applied Spectroscopy, 50 (7), 199–219. Chen YR, Nguyen N, Park B, Chao K (1998a) Intelligent on-line training and updating of automated poultry inspection system. ASAE Paper No. 983047, ASAE, St Joseph, MI, USA. ChenYR, Nguyen M, Park B (1998b) Neural network with principal component analysis for poultry carcass classification. Journal of Food Process Engineering, 21 (5), 351–367. Chen YR, Hruschka WR, Early H (1998c) On-line inspection of poultry carcasses using visible/near-infrared spectrophotometer. Proceedings of the SPIE, The International Society of Optical Engineering, 3544, 146–155. Chen YR, Park B, Huffman RW, Nguyen M (1998d) Classification of on-line poultry carcasses with back propagation neural networks. Journal of Food Process Engineering, 21, 33–48. Chen YR, Hruschka WR, Early H (2000) A chicken carcass inspection system using visible/near-infrared reflectance: in plant trials. Journal of Food Process Engineering, 23 (2), 89–99. Clarke JK, Allan GM, Bryson DG, Willians W, Todd D, Mackie DP, McFerran JB (1990) Big liver and spleen disease of broiler breeders. Avian Pathology, 19, 41–50. Daley W, Rao T (1990) Color vision for industrial inspection. In Proceedings of Machine Vision Association of Society of Manufacturing Engineers, MS90-600. Daley W, Soulakos C, Thomson C, Millet R (1988) A novel application: machine vision inspection, grading, and identification of chicken parts. In Proceedings of Robotics and Vision ’88, Society of Manufacturing Engineers, Dearborn, Michigan, USA. Daley W, Carey R, Thompson C (1994) Real-time color grading and defect detection of food products. Proceedings of the SPIE, The International Society of Optical Engineering, 2345, 403–411. Domermuth CH, Harris JR, Gross WB, DuBose RT (1978) A naturally occurring infection of chickens with a hemorrhagic enteritis/marble spleen disease type of virus. Avian Diseases, 23 (2), 479–484. Evans MD, Thai CN, Grant JC (1997) Computer control and calibration of a liquid crystal tunable filter for crop stress imaging. ASAE Paper No. 973141, ASAE, St Joseph, MI, USA. Fairchild MD (1998) Color Appearance Models. Reading: Addison Wesley. Favier J, Ross DW, Tsheko R, Kennedy DD, Muir AY, Fleming J (1998) Discrimination of weeds in brassica crops using optical spectral reflectance and leaf texture analysis. Proceedings of the SPIE,The International Society of Optical Engineering 3543, 311–318. Hruschka WR (1987) Data analysis: wavelength selection methods. In Near-Infrared Technology in Agricultural and Food Industries (Williams P, Norris. K, eds). St Paul: American Association of Cereal Chemists, pp. 35–55. Hsieh C, Chen YR, Dey BP, Chan DE (2002) Separating septicemic and normal chicken livers by visible/near-infrared spectroscopy and back-propagation neural networks. Transactions ASAE, 45 (2), 459–469.

References 185

Ibarra JG, Tao Y, Newberry L, Chen YR (2002) Learning vector quantization for color classification of diseased air sacs in chicken carcasses. Transactions ASAE, 45 (5), 1629–1635. Jang JR (1993) ANFIS: Adaptive-Network-based Fuzzy Inference System. IEEE Transactions on Systems, Man, & Cybernetics, 23(3), 665–683. Kurse FA, Lefkoff AB, Boardman JB, Heidebrecht KB, Shapiro AT, Barloon PJ, Goetz AFH (1993) The spectral image processing system (SIPS) – interactive visualization and analysis of imaging spectrometer data. Remote Sensing of Environment, 44 (1), 145–163. LiuY, ChenYR (2001)Analysis of visible reflectance spectra of stored, cooked, and diseased chicken meats. Meat Science, 58 (4), 395–401. Liu Y, Fan X, Chen YR, Thayer DW (2003) Changes in structure and color characteristics of irradiated chicken breasts as a function of dosage and storage time. Meat Science, 63 (3), 301–307. Mao C, Heitschmidt J (1998) Hyperspectral imaging with liquid crystal tunable filter for biological and agricultural assessment. Proceedings of the SPIE, The International Society of Optical Engineering, 3543, 172–181. Mazer AS, Martin M, Lee M, Solomon JE (1988) Image processing software for imaging spectrometry analysis. Remote Sensing of Environment, 24 (1), 201–210. Miller BK, Delwiche MJ (1989) A color vision system for peach grading. Transactions ASAE, 34 (4), 1484–1490. Mountney M (1987) US Department of Agriculture standards for processed poultry and poultry products. In The Microbiology of Poultry and Meat Products (Cunningham FE, Cox NA, eds). New York: Academic Press, Ch. 6. Muir AY (1993) Machine vision and spectral imaging. Agricultural Engineering, 48 (4): 124. Nauck D, Kruse R. (1995) NEFCLASS – a neuro-fuzzy approach for the classification of data. Proceedings of the Association for Computing Machinery Symposium On Applied Computing, Nashville, 26–28 Feb. New York: ACM Press. OSHA (1999) Chicken disassembly – ergonomic considerations. http://www.oshaslc.gov/SLTC/poultryprocessing. US Department of Labor, Washington, DC. Park B, Chen YR (1994a) Intensified multi-spectral imaging system for poultry carcass inspection. Transactions ASAE, 37 (6), 1983–1988. Park B, Chen YR (1994b) Multispectral image textural analysis for poultry carcasses inspection. ASAE Paper No. 946027, ASAE, St Joseph, MI, USA. Park B, Chen YR (1998) Real-time multispectral image processing for poultry inspection. ASAE Paper No. 983070, ASAE, St Joseph, MI, USA. Park B, Chen YR, Whittaker AD, Miller RK, Hale DS (1994) Neural network modeling for beef sensory evaluation. Transactions ASAE, 37 (5), 1547–1553. Park B, Chen YR, Huffman RW (1995) Integration of visible/NIR spectroscopy and multispectral imaging for poultry carcass inspection. Proceedings of the SPIE, The International Society of Optical Engineering, 2345, 162–171. Park B, Chen YR, Nguyen M, Hwang H (1996a) Characterizing multispectral images of tumorous, bruised, skin-torn, and wholesome poultry carcasses. Transactions ASAE, 39 (5), 1933–1941.


Park B, Chen YR (1996b) Multispectral image co-occurrence matrix analysis for poultry carcasses inspection. Transactions ASAE, 39 (4), 1485–1491. Park B, Chen YR, Chao K (1998a) Multispectral imaging for detecting contamination in poultry carcasses. Proceedings of the SPIE, The International Society of Optical Engineering, 3544, 110–120. Park B, Chen YR, Nguyen M (1998b) Multi-spectral image analysis using neural network algorithm for the inspection of poultry carcasses. Journal of Agricultural Engineering Research, 69, 351–363. Park B, Lawrence KC, Windham WR, Buhr RJ (2002) Hyperspectral imaging for detecting fecal and ingesta contaminants on poultry carcasses. Transactions ASAE, 45 (6), 2017–2026. Park B, Lawrence KC, Windham WR, Smith DP, Feldner PW (2003). Machine vision for detecting internal fecal contaminants of broiler carcasses. ASAE Paper No. 033051, ASAE, St Joseph, MI, USA. Precetti CJ, Krutz GW (1993) Real-time color classification system. ASAE Paper No. 933002, ASAE, St Joseph, MI, USA. Richards JA, Jia X (1999) Remote Sensing Digital Image Analysis. Berlin: Springer-Verlag. Sakar N, Wolfe RR (1985) Feature extraction techniques for sorting tomatoes by computer vision. Transactions ASAE, 28 (3), 970–979. Saputra D, Payne FA, Lodder RA, Shearer SA (1992) Selection of near-infrared wavelengths for monitoring milk coagulation using principle component analysis. Transactions ASAE, 35 (5), 1597–1605. Schat KA (1981) Role of the spleen in the pathogenesis of Marek’s disease. Avian Pathology, 10, 171–182. Swatland HJ (1989) A review of meat spectrophotometry (300 to 800 nm). Canadian Institute of Food Science and Technology Journal, 22 (4), 390–402. Tao Y, Morrow CT, Heinemann PH, Sommer JH (1990) Automated machine vision inspection of potatoes. ASAE Paper No. 903531, ASAE, St Joseph, MI, USA. Tao Y, Heinemann PH, Varghese Z, Morrow CT, Sommer III HJ (1995) Machine vision for color inspection of potatoes and apples. Transactions ASAE, 38 (5), 1555–1561. Tao Y, Shao J, Skeeles JK, Chen YR (1998) Spleen enlargement detection of eviscerated turkey by computer vision. Proceedings of the SPIE, The International Society of Optical Engineering, 3544, 138–145. Tao Y, Shao J, Skeeles K, Chen YR (2000) Detection of splenomegaly in poultry carcasses by UV and color imaging. Transactions ASAE, 43 (2), 469–474. Throop JA, Aneshansley DJ (1995) Detection of internal browning in apples by light transmittance. Proceedings of the SPIE, The International Society of Optical Engineering, 2345, 152–165. Tsuta M, Sugiyama J, Sagara Y (2002) Near-infrared imaging spectroscopy based on sugar absorption for melons. Journal Agricultural Food Chemistry, 50 (1), 48–52. USDA (1984) A review of the slaughter regulations under the Poultry Products Inspection Act. Regulations Office, Policy and Program Planning, FSIS, USDA, Washington, DC. USDA (1985) Meat and Poultry Inspection. Committee on the Scientific Basis of the Nation’s Meat and Poultry Inspection Program. Washington, DC: National Academy Press.

References 187

USDA (1996a) Key Facts: Economic impact analysis. USDA, FSIS, HACCP Rule-Economic Analysis. Washington, DC: USDA/FSIS. USDA (1996b) Pathogen Reduction; Hazard Analysis and Critical Control Point (HACCP) Systems, Final Rule. Fed. Reg. 61: 38805–38855. USDA (2005) Agricultural Statistics, National Agricultural Statistics Service, Washington, DC. Wen Z, Tao Y (1998) Fuzzy-based determination of model and parameters of dualwavelength vision system for on-line apple sorting. Optical Engineering, 37 (1), 293–299. Williamson SJ, Cummins HZ (1993) Light and Color in Nature and Art. New York: Wiley. Windham WR, Lawrence KC, Park B, Buhr RJ (2003a) Visible/NIR spectroscopy for characterizing fecal contamination of chicken carcasses. Transactions ASAE, 46 (3), 745–751. Windham WR, Smith DP, Park B, Lawrence KC, Feldner PW (2003b) Algorithm development with visible/near-infrared spectra for detection of poultry feces and ingesta. Transactions ASAE, 46 (6), 1733–1738. Yang CC, Chao K, Chen YR, Kim MS (2004) Application of multispectral imaging for identification of systemically diseased chicken. ASABE Paper No. 043034. Yang CC, Chao K, ChenYR, Kim MS, Early HL (2006) Simple multispectral image analysis for systemically diseased chicken identification. Transactions ASAE, 49 (1), 245–257.

Quality Evaluation of Seafood Murat O. Balaban1 , Asli Z Odaba¸si1 , Sibel Damar1 and Alexandra C.M. Oliveira2 1 University of Florida, Food Science and Human Nutrition Department,

PO Box 110370, Gainesville, FL 32611-0370, USA 2 Fishery Industrial Technology Center, University of Alaska Fairbanks, Kodiak, AK 99615, USA

1 Introduction Quality attributes of seafood include appearance (size, shape, color), smell, taste, nutritional aspects, and safety-related properties. Machine vision (MV) can potentially evaluate all these attributes. Smell and taste are the most difficult to evaluate with MV, although volatile attributes can be related to color for analysis (Rakow and Suslick, 2000; Suslick and Rakow, 2001) by inducing color changes in an array of dyes and permitting visual identification. Nutrition can also be evaluated as far as some proximate composition components are concerned (such as moisture content and fat) using, for example, near infrared (Wold and Isakkson, 1997). Pinbones, shell fragments, and other undesirable matter can also be recognized by MV (Graves, 2003). Direct measurement of safety (microbial, chemical, metal fragments, etc.) is currently difficult to measure using MV. Visual attributes of seafood will be discussed in this chapter. They include size, shape, and color. Brief literature examples will be given for each, and some of the research performed in our laboratory will be presented.

2 Visual quality of seafood 2.1 Size 2.1.1 Literature Arnarson (1991) describes a system to sort fish and fish products by machine vision. The difficulties of sorting fish are listed as: fast speed requirements, number of species, variation of the size and shape of each species, variation of the optical characteristics of Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


8

190 Quality Evaluation of Seafood

each fish, the elastic nature of fish, and the harsh environment in factories. The length measurement of cod was accomplished by measuring the distance between the middle of the tail and the top of the head of a straight fish. The accuracy of length measurement was ± 0.9 cm, independent of size and orientation. Bent fish are allowed. The algorithm detected each fish, drawing a rectangle around it. It then detected the head, the tail, and the belly, and drew a line from the middle of the tail to the top of the head, taking into account the position of the belly. Each fish required an analysis time of between 0.6 and 0.8 seconds. The most accurate method to measure free-swimming fish is to use a stereo video system (Batty, 1989; Champalbert and Direach-Boursier, 1998), where the space in front of two cameras can be calibrated in three dimensions using landmark points. Freeswimming fish can then be viewed from different angles and orientations. However, these sophisticated systems require a great deal of computing power, and work better on relatively large fish. Martinez-Palacios et al. (2002) used a video camera and recorder to measure the length of larval and juvenile white fish (Chirostoma estor estor). Juvenile fish were placed in a Petri dish with water, and images were taken from the top with a calibration grid of 1-mm lines. The dry weight of the fish was correlated to length using a logarithmic equation with an r 2 = 0.99. The average weight estimation error was 2 percent. Oysters are generally sold by volume. Accurate oyster grading is critical for pricing, and oysters are typically graded and sorted by humans before and after shucking. This is a labor-intensive, time-consuming, and subjective process. Oysters do not have a regular shape (Diehl et al., 1990), and the dimensions and the overall thickness of oysters, the thickness of their shells, and the amount of meat vary depending on the species, location, age, pre- or post-spawning, and individual oyster (Li, 1990; Hamaoka and Sasaki, 1992). It is clearly desirable to accurately predict the volume or weight of oysters for sorting and pricing. Machine vision, being a fast, accurate, and non-contact method of grading various food products, can also be applied to oysters. Parr et al. (1994) developed a machine-vision based sorting and grading system for oyster meat. In a continuous system, one oyster could be graded into one of three grades in 2 seconds. Tojeiro and Wheaton (1991) developed a system, based on a black-and-white video camera and a mirror, to obtain the top and side views of an oyster simultaneously. Further, they also developed software to determine the ratio of thicknesses about 1.5 cm from each end to decide on the hinge side. The method oriented 233 oysters with a correct rate of 98.2 percent. Li and Wheaton (1992) used a Wheaton shucking machine to trim the hinge-ends of oysters and obtained images using a video camera. A pattern recognition technique was used to locate oyster hinge lines, with an error rate of 2.5 percent. In 2002, So and Wheaton published results of their latest software development efforts to automate oyster hinge-line detection using machine vision. This time a color camera was used. The software calculated circularity, rectangularity, aspect ratio, and Euclidian distance to recognize the hinge from other dark objects on the hinge end of the oyster. Lee et al. (2001) used a laser-line based method to predict the volume of oyster meat. The thickness information was gathered by the shape of the laser line on the meat. The predicted volume was compared to the experimentally determined volume, where the correlation coefficient was 0.955.

Visual quality of seafood 191

Weight (g)

15

10 intact head off peeled tail off

5

1

2

3

4

5 6 7 8 View pixels (thousands)

9

10

11

Figure 8.1 Weight of different white shrimp forms vs view area obtained from a machine vision system.

2.1.2 Determination of shrimp weight, count, and uniformity ratio

Quality evaluation practice for shrimp involves a trained inspector who weighs a shrimp sample, counts the number of shrimp, and calculates the count (number/unit weight) and uniformity ratio UR (a weight ratio of the largest 10 percent of shrimp to the smallest 10 percent of shrimp). The inspector looks for visible defects such as melanosis (black spots formed by enzymatic activity), foreign material, shell parts, and broken pieces. This subjective practice can be automated. Luzuriaga et al. (1997) developed calibration relationships of the view area obtained by MV vs the weight of intact, headless, peeled tail-on, and peeled tail-off white shrimp (Penaeus setiferus). Head-on, non-frozen shrimp were placed individually in the light box described by Balaban et al. (1994), and an image was acquired. The view area of the shrimp in pixels was measured. The shrimp was then weighed; 100 shrimp were processed in this way. The same procedure was repeated three times for the same 100 shrimp after they had been deheaded, peeled, and the tail removed, respectively. The results are shown in Figure 8.1. Several equations were tested to correlate the weight to the view area in pixels. The best fits are shown in Table 8.1. It is evident that for a shrimp that is not split, weight can be accurately predicted by view area. Once weight is determined, then count and uniformity ratio are easy to calculate. In industrial practice, one issue would be whether shrimp were touching or partially blocking each other. There are applications of estimating the weight of shrimp by machine vision in the industry – for example, the Marel (Reykjavik, Iceland) Model L-10 “Vision Weigher” for shrimp processing. Once calibrated, the system estimates the weight of a shrimp from its view area. 2.1.3 Oyster volume

Damar et al. (2006) experimentally measured volumes (overall, shell, meat) of oysters from Florida, Texas, (Crassostrea virginica), and Alaska (Crassostrea gigas) using the Archimedes principle. Using a machine vision system, the top and side view images of whole oysters were captured (Figure 8.2), and the actual view areas were calculated by calibrating pixel area with that of a known square.


Table 8.1 Experimentally determined and estimated total weight, count, and uniformity ratio values for different forms of white shrimp. y = weight (g), x = view area (pixels)

Intact (n = 97)

Headless (n = 99)

Peeled (n = 101)

Tail off (n = 100)

0.508 1.96 × 10−7 1.419 0.964

2.182 3.40 × 10−5 2.011 0.965

0.874 2.05 × 10−6 1.767 0.981

2.037 6.15 × 10−9 2.474 0.968

Total weight (g) Experimental Calculated

867.0 867.5

593.6 592.2

509.4 505.6

461.1 460.2

Count/kg Experimental Calculated

112.0 111.8

166.8 167.2

198.2 199.8

216.9 217.4

Uniformity ratio Experimental Calculated

3.11 2.98

3.17 2.89

3.23 3.12

3.32 2.93

0.980 1.57 × 10−5 0.964

0.831 1.94 × 10−5 0.972

0.116 2.21 × 10−5 0.980

0.371 2.63 × 10−5 0.977

Total weight (g) Experimental Calculated

867.0 865.7

593.6 593.5

509.4 508.7

461.1 462.0

Count/kg Experimental Calculated

112.0 112.0

166.8 166.8

198.2 198.7

216.9 216.5

Uniformity ratio Experimental Calculated

3.11 2.95

3.17 3.08

3.23 3.22

3.32 3.22

Fit: y = a + bx c

Fit: y = a + bx 1.5

a= b= c= R2 =

a= b= R2 =

The oyster image was divided into an even number of volume slices of equal thickness between points p1 and p2. The sum of all the volume slices would give the total volume. Coordinates of points a, b, c, and d were determined from the image. The distance between a and b along the X axis, and the distance between c and d along the Y axis, are shown in Figure 8.2. Points c and d were assumed to be at the midpoint of points a and b along the X axis. Therefore: ax + b x 2 dx = cx cx =

(8.1)

The cross-sectional area at each volume slice (shown in Figure 8.2) was found by fitting a cubic spline to points a, c, and b, and another to points a, d, and b. The cross-sectional area formed by these two curves was calculated as: Cross-sectional area =

5 (bx − ax )(d y − c y ) 8

(8.2)


1 X

Y ax

c

a

p1 p2 bx

a

b

b 2

Z Y

cy

c

dy

d

d 3

Z

X

4 Figure 8.2 Determination of oyster volume using cubic splines.

These cross-sectional areas were integrated along the Z axis using Simpson’s method: ⎛ ⎞ ⎛ ⎞ n n h Volume = ⎝ 4 areai ⎠ + ⎝ 2 areai ⎠ (8.3) 3 i=1,i is even i=1,i is odd

where n is the number of cross sections, and h = (p2z − p1z )/n. Texas oysters had an average experimental volume of 66.52 ± 18.90 cm3 and an average calculated volume of 67.33 ± 19.70 cm3 . Figure 8.3 shows the predicted and experimental volumes of Texas oysters (r 2 = 0.93). Total oyster volume, meat volume, and meat weight were also correlated with the view areas. However, more research is needed in this area to validate this method in oysters from different locations and seasons, and with different spawning status. This method can potentially be used to sort oysters on a conveyor line.

2.2 Shape 2.2.1 Literature Williams et al. (2006) used underwater video from fish-farm cages to detect salmon in images collected. The 256 gray-level images were contrast-enhanced, and the background was removed by segmentation. An active shape model technique was applied: a collection of labeled points are determined to define boundaries of a specified shape.


160

Calculated volume (cm3)

140 120 100 Texas

80

XY

60 40 20 0 0

50

100

150

Real volume (cm3) Figure 8.3

Comparison of oyster volume calculated by cubic splines and measured experimentally.

During training, statistical variation between the points is determined. A model representing the average appearance in the training set is obtained from the mean values of each point. This results in a point distribution model with a number of parameters that can be altered during a search to identify a shape even when it is deformed. From 125 initial fish images, 65 (or 52 percent) were correctly matched using the salmon shape model. Shadows, and fish swimming towards or away from the camera, created problems, as well as segmentation inaccuracies. In the fisheries and seafood area, prawns can be automatically graded and packaged into a single layer with the same orientation by combining machine vision and robotics (Kassler et al., 1993). Morphological and spectral features of shrimp can be determined to find the optimum location for removal of shrimp heads (Ling and Searcy, 1989). Fish species can be sorted according to the shape, length, and orientation of the fish in a processing line (Strachan et al., 1990; Strachan, 1993). Digital image processing of fall chum salmon was used to find an objective criterion to predict the flesh redness from the spawning coloration (Hatano et al., 1989). 2.2.2 Evaluation of rigor mortis in sturgeon A new method to determine the onset and resolution of rigor in cultured Gulf sturgeon (Ancipenser oxyrynchus desotoi) was developed using analysis of video images (Oliveira et al., 2004). Insight into the progress of rigor through the fish body was provided. At 10 different time intervals, from 0 to 67 hours after death, fish were temporarily secured to the edge of a table by the head, with the body free to droop, and video images were taken (Figure 8.4). The extent of deflection of various points along the body length was analyzed. New parameters based on maximum deflection and integral deflections were developed. The displacements in the horizontal and vertical directions of various points along the spine were measured by this method. Therefore, the times at which a particular point entered rigor, reached maximum rigor, and rigor was dissolved could be observed (Figures 8.5


85 cm 3

4 Screw

1 Point (0,0) 2 60 cm

120 cm

5

50 cm 6

Figure 8.4 Experimental set-up for the measurement of rigor mortis in Gulf sturgeon.

13

12

0h

Y Values (inches)

13 h

21 h

Resolution @ 67 h

11

Onset @ 27 h

59 h 10

36 h

52 h

46 h

9

Rigor max. @ 31 h

8 12

13

14

15

16

X Values (inches) Figure 8.5 Movement of a point on the spine 66% of the length of the fish from the head, over time.

and 8.6). For example, a point along the spine 66 percent of the total distance from the head entered rigor 27 hours after death, maximum rigor was reached after 31 hours, and rigor was dissolved at the sixty-seventh hour after death (Figure 8.5). The tail also entered rigor 17 hours after death; however, maximum rigor was reached after 46 hours, while dissolution was again at the sixty-seventh hour (Figure 8.6).

2.3 Color Color is one of the most important visual quality attributes of foods. The first purchasing decision regarding acceptance or rejection of a food generally depends on its color.


23 22

Resolution @ 67 h 0h

21 h

21

Y Values (in)

20

Onset @ 27 h

13 h 59 h

19

36 h 31 h

18 17 52 h 16 Rigor max. @46 h

15 14 12

Figure 8.6

14

16

18 X Values (in)

20

22

Movement of the tail of the fish, over time.

Machine vision has unique capabilities in measuring color, especially of foods of nonuniform color and surface characteristics. This section provides brief examples from the literature regarding color evaluation of seafood. 2.3.1 Color space

Color machine vision systems generally capture images in the red, green, blue (RGB) color system as 24-bit images. Each color axis is allocated 8 bits, resulting in 256 different values. This gives 16.7 million possible color combinations (256 × 256 × 256). Since it is difficult to handle 16.7 million colors it was decided to reduce the number of colors in the color space, and this was done by dividing each color axis (red, green, blue) into 4, 8, or 16 (Luzuriaga et al., 1997). In the three-dimensional color space this resulted in 64, 512, or 4096 “color blocks” (Figure 8.7). Any color in the block was represented by the center color of that block. This effectively reduced the number of colors from 16 million to a manageable number. It was expected that some loss of information would occur. Indeed, for the 64 color-block system, some “averaging” of the colors occurred (Figure 8.8), resulting in patchy and artificial colors. However, the 4096 color-block system was visually indistinguishable from the real image. The software for most machine vision applications can also convert the color information from one system to another. Typical color systems include XYZ, Hue–Saturation–Lightness, Munsell, RGB, L-a-b. 2.3.2 Shrimp color

Luzuriaga et al. (1997) objectively measured melanosis levels in white shrimp (Penaeus setiferus) stored on ice for up to 17 days, to correlate these with the evaluation grades


Figure 8.7 Formation of “color blocks’’ by dividing the RGB axes into different parts. (A color version can be viewed at http://books.elsevier.com/companions/9780123736420)

of a human expert, and to quantify the color changes occurring in the shrimp stored on ice for up to 17 days. The inspector visually evaluated each sample and graded it for melanosis on a scale from 0 = none to 10 = high (Otwell and Marshall, 1986). A range of values, such as 1–2 or 5–6, was assigned by the inspector. As soon as the inspector’s evaluation was complete, the shrimp were analyzed for melanosis by the MV system to measure the percentage area with melanosis. Images of both sides of the shrimp were analyzed, and melanosis values were averaged. The trained inspector identified the black and the dark areas as melanotic. The RGB colors of the dark areas identified by the trained inspector were determined. These dark color blocks were added to the melanosis analysis. Six color blocks from the 64 used in this system were chosen as candidates to be included in melanosis calculations. Table 8.2 shows the specifications of these colors. When these colors were included in the analysis of melanosis by MV, the correlation with the inspector’s evaluation of the shrimp was r 2 = 0.68 (Figure 8.9). The change of the melanotic colors over storage time is shown in Figure 8.10. 2.3.3 Color evaluation of carbon-monoxide treated seafood

Carbon monoxide (CO) is known to bind to hemoglobin and myoglobin, resulting in a cherry-red color (Kristinsson et al., 2006). Excess use of CO may result in an unnatural color in seafood products such as tuna, mahi mahi, tilapia, and catfish. Balaban et al. (2006) studied the possibility of “improving” the color of tuna and mahi mahi, starting with inferior-quality fish. An MV system was used to quantify the color of the fish before and after treatments, and during refrigerated storage. Since tuna has a fairly uniform color, a colorimeter can be used to quantify its color. The advantage of the


Original picture

64 color blocks

512 color blocks

4096 color blocks

Figure 8.8 Comparison of 64, 512, and 4096 color blocks with the original image. (A color version can be viewed at http://books.elsevier.com/companions/9780123736420) Table 8.2 Specification of melanotic and light color blocks. Color block#

1 2 4 8 46 56

Melanotic colors

Light colors

R value

G value

B value

Color block (no.)

R value

G value

B value

32 96 160 160 32 96

32 32 96 32 96 96

32 32 32 32 32 32

62 61 60 58 57 7

224 224 224 224 224 32

224 224 224 160 160 96

160 96 32 160 96 224

MV system becomes evident when analyzing the color of mahi mahi, which has a dark muscle strip surrounded by light muscle. When using a colorimeter, the number and position of the locations at which the colorimeter is to be placed when making the measurements, and the aperture size of the colorimeter, affects the accuracy of


10

45 Human expert grade 40

Melanosis grade

8

MV grade

35

% Area as melanosis

30

6

25 4

20 15

2 10 0

5 0

3

7

9

13

15

17

Storage time on ice (days)

% area

Figure 8.9 Comparison of melanosis grade from human inspector, MV system, and the melanotic areas of shrimp stored on ice.

25 20 15 10 5 0 0 3 7 9 13 15 17 Storage (days)

56

46

8

2

4

1

Color block

% area

(a)

50 40 30 20 10 0

0 3 7 9 13 15

17

62

61

60

58

57

7

6

5

Color block

Storage (days) (b) Figure 8.10 Change of “dark’’ (melanotic) and “light’’ colors of shrimp over storage time.

the average color values calculated. The MV can analyze all the pixels of the sample, eliminating the limitations of sampling frequency, size and location mentioned above. The authors found that it was possible to start with inferior-quality fish, treat it with 100 percent CO, and “improve” its color to be comparable to or better than fish of good


Mahi Mahi (enhancement with CO) a*

b*

L*

60

50

Values

40

30 fresh

1 week on ice

100% CO

20

10

0 Figure 8.11 Quantification of color changes in original, refrigerated (1 week), and refrigerated (1 week) then CO-treated mahi mahi, by MV. (A color version can be viewed at http://books.elsevier.com/ companions/9780123736420)

quality. An example of the images and resulting L*, a* and b* values for mahi mahi are shown in Figure 8.11. 2.3.4 Sorting whole salmon by skin color

Pink salmon (Oncorhynchus gorbuscha) is sorted visually and grades are assigned to determine price. Typical fish with the grades from A to F (decreasing value) are shown in Figure 8.12. The typical indices of quality are the brightness of the skin color, and the lack of “watermarking” on the skin of the fish. Since fatigue and resulting errors occur on human inspection lines, it is desirable to apply MV sorting of the intact fish by skin appearance. A study was initiated in 2003 into the accurate sorting of pink salmon (Oliveira et al., 2006a). An expert evaluated 94 fish and assigned a grade each. This initial grading resulted in 21 fish of AB grade, 23 fish of CD, 8 fish of DE, 32 fish of E, and 10 fish of F grade. After grading, each fish was placed in a light box, similar to that described in Luzuriaga et al. (1997) but with larger dimensions to accommodate whole fish, and an image was acquired with MV. The first approach was to calculate the average L*, a*, and b* of each image by averaging these values of all pixels of each fish. The average L* values were compared to the grades assigned by an expert. Figure 8.13 shows that there was much variation in the average L* value of the whole fish to allow for accurate prediction of the grade. The next trial was to quantify the percentage of the total fish surface with an L* value greater than a threshold value between 60 and 90 (Balaban et al., 2005). This was accomplished using LensEye software (Gainesville, FL). Averages were taken for each grade (Figure 8.14). A threshold level of L* > 85 was chosen since it had the smoothest line. The percentage surface area with L* > 85 for each fish was plotted against the


AB

CD

DE

E

F

Figure 8.12 Grading of pink salmon by skin color with emphasis on “watermarking’’. (A color version can be viewed at http://books.elsevier.com/companions/9780123736420)

Average L* value, whole fish

100

90

80

70

60

50

1

2

3

4 5 6 7 Experimental grade

8

9

10

Figure 8.13 L* values averaged over the whole surface of pink salmon with different human expert assigned grades.


% surface area with L theshold

80 70

Threshold

60

60 65 70 75 80 85 90

50 40 30 20 10 0

1

2

3


8

9

10

Figure 8.14 Percentage of the total surface area of whole pink salmon assigned different grades by human expert. Different threshold results are shown.

70

Y 54.38 4.09 X % Surface with L 85

60

R2 0.61

50 40 30 20 10 0

1

2

3


8

9

10

Figure 8.15 Linear regression of percentage surface area with L* < 85 vs human evaluation of experimental grade.

experimental grade (Figure 8.15). The objective was to analyze the image of a fish obtained from MV, calculate the percentage surface with L* > 85, locate this value on the Y axis of Figure 8.15, and predict the grade using the regression line. However, it was obvious that the scatter of the data precluded accurate prediction of the grade. Therefore, it was decided to select a region of interest (ROI) on each fish. This was chosen as a rectangular area bounded by the lateral line at the top, behind the gill plate towards the head, the pectoral fin at the bottom, and the end of the dorsal fin towards the tail (Figure 8.12). It was expected that this area would have less variation and more significance regarding watermarking. The percentage surface area of the ROI having pixels with L* values lower than a threshold (between 60 and 90) are shown in Figure 8.16. The threshold value of L* < 80 was selected because it had the least


100

Percent of ROI area

80

60 AB CD DE

40

E F 20

0 L 60

L 65

L 70

L 75

L 80

L 85

L 90

20 Figure 8.16 Different threshold values of salmon grades based on percentage area < threshold L* value. Average L, a, b values

90 80 70 60 AB

50

CD

40

DE

30

E F

20 10 a* average 0 10

L* average

b* average

Figure 8.17 Average L*, a*, b* values, determined by machine vision, of each grade of salmon in the regions of interest shown in Figure 8.12.

amount of variation (error bars in Figure 8.16). The average L*, a*, and b* values of the ROIs for each grade are shown in Figure 8.17. The correlation of the average L* value of the ROI and the percentage surface of ROI with L* < 80 is shown in Figure 8.18. The latter parameter was chosen, since it had a larger spread. Finally, an iterative procedure was performed where the average percentage ROI surface with L* < 80 was taken for each grade. Between grades AB and CD, the


100 AB CD DE E F

Average L* of ROI area

90

80

70

60

50 0

20

60 40 80 Percent of ROI area L* 80

100

Figure 8.18 Relationship between average L* of the regions of interest (ROI) shown in Figure 8.12 for different grades of salmon, and the percentage of the ROI with L* < 80.

Table 8.3 Human expert and machine vision classification of whole pink salmon by skin color. MV prediction →

AB

CD

DE

E

F

Human grade (total fish) AB (21) CD (23) DE (8) E (32) F (10) Total (94)

18 11 0 0 0 29

2 7 2 5 0 16

1 5 1 9 0 16

0 0 5 5 3 13

0 0 0 13 7 20

difference in these averages was divided in half, and this was taken as the separation level between the grades AB and CD. The same was applied to the other grades. Next, each fish was reclassified by its L* < 80 value by moving it into the appropriate grade range. This was repeated until no fish moved between grade ranges. The result is shown in Table 8.3. It is important to note that many fish were misclassified by the human expert in the mid-grade DE, based on the results in Table 8.3. For very high-grade (AB) or the very low-grade (F) fish, the human and MV estimations of grade were similarly accurate. The misclassified fish were re-examined by the human expert, and the new grades assigned by the MV system were confirmed. These results are encouraging for the use of MV in efficiently grading whole salmon, on a conveyor belt, by skin color. 2.3.5 Comparison of MV and colorimeter evaluation of sturgeon color

Oliveira and Balaban (2006b) compared the color readings of a hand-held colorimeter with a MV system in measuring the color of Gulf of Mexico sturgeon fillets from fish fed different diets, and refrigerated for up to 15 days (Figure 8.19). The L*a*b* values were measured at days 0, 5, 10, and 15 using both instruments, and E values


1/2 L

Fillet width W

1/3 F L

1/2 W

Chromameter Fillet length F

Machine vision center slice

Figure 8.19 Determination of color of sturgeon fillets. Location of colorimeter measurements (above), and the machine vision region. (A color version can be viewed at http://books.elsevier.com/ companions/9780123736420)

calculated to allow comparison of results. The E value measures the “total” color change, described as: E = (Lo − Li )2 + (ao − ai )2 + (bo − bi )2 (8.4) where: the subscript o refers to the values at time 0, and i refers to values at 5, 10, or 15 days. Statistical analysis indicated that there were no significant differences in E values from the hand-held colorimeter or machine vision between either treatments or storage days (P < 0.05). E values were significantly different (P < 0.05) between instruments, except for day 0. The large differences in E for the colorimeter between day 0 and day 1 did not reflect the mild color changes over time visually observed from pictures. The authors concluded that machine vision had the ability to measure color with high spatial resolution, thus it could outperform other colorimeters when recording and estimating subtle and non-uniform color changes in foods. 2.3.6 Combining color with other quality parameters The advantage of evaluating colors not as an average but as discrete values allows different types of analyses, such as discriminant function and neural network methods. In a storage study, Korel et al. (2001a) used a color machine vision system (MV) to monitor the changes in the color of tilapia (Oreochromis niloticus) fillets dipped in sodium lactate solutions (0%, 4%, 8% (v/v)). The use of MV allowed for the percentage of each of the color blocks in a 64-color block system to be calculated in addition to the reporting of the average L*a*b* values. The authors selected those color blocks that represented the color of areas that made up at least 5 percent of the fillet surface. Twenty color blocks selected were used in a discriminant function analysis to classify


6 Treatments

5

Control Lactate 4% Lactate 8%

Discriminant function 2

4 3 2 1 0 1 2 3 4 4

3

2

1

0

1

Discriminant function 1 Figure 8.20

2

3

4

(Ellipses 95% confidence area)

Discriminant function analysis of tilapia color for all treatments at 1.7◦ C, based on color data.

the observations into one of the lactate treatment groups. The corresponding overall correct classification rate was 82 percent (Figure 8.20). For each lactate treatment, the color block data were classified into storage time groups and correct classification rates between 56–80 percent were observed. These rates improved significantly when electronic nose data were combined with the color block data: 100 percent of the observations were correctly classified into their respective storage time group. The authors recommended the use of such an approach where MV measurements of color and electronic nose data are combined to locate the group (defined by storage time) of a tilapia sample, the storage history of which may be unknown. In another study (Korel et al., 2001b), raw and cooked catfish (Ictalurus punctatus) fillets were evaluated with MV and electronic nose throughout storage. Similar to the tilapia study previously described, correct classification was obtained for all observations when discriminant function analysis was performed on color block and electronic nose data to group samples with respect to storage time (Figure 8.21). It was concluded that MV data, especially when combined with another tool like electronic nose, provide an improvement towards the determination of overall food quality. A similar study with oyster color and e-nose data, analyzed by discriminant function, resulted in similar conclusions (Tokusoglu and Balaban, 2004).

3 Conclusions Seafood is a food commodity that has great variation in shape, size, color and other visual properties when it comes to expected quality attributes. Non-uniform sizes, shapes, surfaces, and colors are common. This constitutes a challenge to the evaluation of parameters by traditional instruments or methods. The visual quality of seafood

References 207

5 4

Discriminant function 2

3 2 1 0 1 2

Sensory scores

3

Fresh Borderline Spoiled

4 5 6

4

2

0

2

4

6

Discriminant function 1 Figure 8.21 Discriminant function analysis of catfish color based on sensory scores.

can be measured by machine vision accurately, in a non-contact, non-destructive, and continuous manner. As data from more research accumulate, and as hardware becomes faster and more affordable, it is expected that MV will find more real-world applications in the quality evaluation of seafood. Combination of machine vision data with other sources, such as electronic nose or near-infrared analysis, will synergistically improve quality evaluation.

References Arnarson H (1991) Fish and fish product sorting. In Fish Quality Control by MachineVision (Pau LF, Olafsson R, eds). New York: Marcel Dekker, pp. 245–261. Balaban M O, Yeralan S, Bergmann Y (1994) Determination of count and uniformity ratio of shrimp by machine vision. Journal of Aquatic Food Product Technology, 3 (3), 43–58. Balaban MO, Kristinsson HG, Otwell WS (2005) Evaluation of color parameters in a machine vision analysis of carbon monoxide-treated fresh tuna. Journal of Aquatic Food Product Technology, 14 (2), 5–24. Balaban MO, Kristinsson HG, Otwell WS (2006) Color enhancement and potential fraud in using CO. In Modified Atmosphere Processing and Packaging of Fish: Filtered Smokes, Carbon Monoxide & Reduced Oxygen Packaging (Otwell WS, Balaban MO, Kristinsson HG, eds). Ames: Blackwell Publishing, pp. 127–140. Batty RS (1989) Escape responses of herring larvae to visual stimuli. Journal of Marine Biological Association of the United Kingdom 69 (3), 647–654. Champalbert G, Direach-Boursier LL (1998) Influence of light and feeding conditions on swimming activity rythms of larval and juvenile turbot: an experimental study. Journal of Sea Research, 40 (3–4), 333–345.


Damar S, Yagiz Y, Balaban MO, Ural S, Oliveira ACM, Crapo CA (2006) Prediction of oyster volume and weight using machine vision. Journal of Aquatic Food Product Technology, 15(4), 5–17. Diehl KC, Awa TW, Byler RK, van Gelder MF, Koslav M, Hackney CR (1990). Geometric and physical properties of raw oyster meat as related to grading. Transactions of the ASAE, 33, 1270–1274. Graves, M. (2003). X-ray bone detection in further processed poultry production. In Machine Vision for the Inspection of Natural Products (Graves, M. and Batchelor, B., eds). New York: Springer-Verlag, pp. 421–448 Hamaoka T, Sasaki K (1992) Development for a system for judging the freshness of raw oysters from Hirsoshima using fuzzy reasoning. Japanese Journal of Fuzzy Theory and Systems, 4(1), 65–73. Hatano M,Takahashi K, OnishiA, KameyamaY (1989) Quality standardization of fall chum salmon by digital image processor. Nippon Suisan Gakkaishi, 55 (8), 1427–1433. Kassler M, Corke P, Wong P (1993) Automatic grading and packing of prawns. Computers and Electronics in Agriculture, 9, 319–333. Korel F, D A Luzuriaga, Balaban MO (2001a) Objective quality assessment of raw tilapia (Oreochromis Niloticus) fillets using electronic nose and machine vision. Journal of Food Science, 66 (7), 1018–1024. Korel F, Luzuriaga, DA, Balaban MO (2001b) Quality evaluation of raw and cooked catfish (Ictalurus punctatus) using electronic nose and machine vision. Journal of Aquatic Food Product Technology, 10 (1), 3–18. Kristinsson HG, Balaban MO, Otwell WS (2006) The influence of carbon monoxide and filtered wood smoke on fish muscle color. In Modified Atmosphere Processing and Packaging of Fish: Filtered Smokes, Carbon Monoxide & Reduced Oxygen Packaging (Otwell WS, Balaban MO, Kristinsson HG, eds). Ames: Blackwell Publishing, pp. 29–53. Lee DJ, Lane RM, Chang GH (2001) Three-dimensional reconstruction for high speed volume measurement. Proceedings of SPIE, 4189, 258–267. Li J (1990) Oyster hinge line detection using digital image processing. Presented during the 1990 International Summer Meeting of the ASAE, June 24–27, Columbus, OH. Li J, Wheaton FW (1992) Image processing and pattern recognition for oyster hinge line detection. Aquacultural Engineering, 11, 231–250. Ling PP, Searcy SW (1989) Feature extraction for a vision based shrimp deheader. Presented during the 1989 International Winter Meeting of the ASAE, December 12–15, New Orleans, LA. Luzuriaga D, Balaban MO, Yeralan S (1997) Analysis of visual quality attributes of white shrimp by machine vision. Journal of Food Science, 62 (1), 1–7. Martinez-Palacios CA, Tovar EB, Taylor JF, Duran GR, Ross LG (2002) Effect of temperature on growth and survival of Chirostoma estor estor, Jordan 1879, monitored using a simple video techniques for remote measurement of length and mass of juvenile fishes. Aquaculture, 209, 369–377. Oliveira ACM, O’Keefe SF, Balaban MO (2004) Video analysis to monitor rigor mortis in cultured Gulf of Mexico sturgeon (Ancipenser oxyrynchus desotoi). Journal of Food Science, 69 (8), E392–397.

References 209

Oliveira, ACM, Crapo, C and Balaban MO (2006a) Grading of pink salmon skin watermarking using a machine vision system. Second Joint Transatlantic Fisheries Technology Conference. October 29–November 1, 2006, Quebec City, Quebec, Canada. P-46, p. 138. Oliveira ACM, Balaban MO (2006b) Comparison of a colorimeter with a computer vision system in measuring color of Gulf of Mexico sturgeon fillets. Applied Engineering in Agriculture, 22 (4), 538–587. Otwell S, Marshall M (1986) Studies on the use of sulfites to control shrimp melanosis (blackspot). Florida Sea Grant College, Technical Paper No. 46, Gainesville, FL, USA. Parr MB, Byler RK, Diehl KC, Hackney CR (1994) Machine vision based oyster meat grading and sorting machine. Journal of Aquatic Food Product Technology, 3 (4), 5–25. Rakow NA, Suslick KS (2000) A colorimetric sensor array for odor visualization. Nature, 406, 710–713. So JD, Wheaton FW (2002) Detection of Crassostrea virginica hinge lines with machine vision: software development. Aquacultural Engineering, 26, 171–190. Strachan NJC (1993) Length measurements of fish by computer vision. Computers and Electronics in Agriculture, 8, 93–104. Strachan NJC, Nesvadba P, Allen A R (1990) Fish species recognition by shape analysis of images. Pattern Recognition, 23 (5), 539–544. Suslick KS, Rakow NA (2001) A colorimetric nose: “smell-seeing”. In Artificial Chemical Sensing: Olfaction and the Electronic Nose (Stetter JR, Pensrose WR, eds). Pennington: NJ Electrochemical Society, pp. 8–14. Tojeiro P, Wheaton F (1991) Oyster orientation using computer vision. Transactions of the ASAE, 34 (2), 689–693. Tokusoglu O, Balaban MO (2004). Correlation of odor and color profiles of oysters (Crassostrea virginica) with electronic nose and color machine vision. Journal of Shellfish Research, 23 (1), 143–148. Williams RN, Lambert TJ, Kelsall AF, Pauly T (2006) Detecting marine animals in underwater video: let’s start with salmon. Proceedings of the 12th Americas Conference on Information Systems, August 4–6, Acapulco, Mexico, pp. 1482–1490. Wold JP, Isakkson T (1997) Non-destructive determination of fat and moisture in whole Atlantic salmon by near-infrared diffuse spectroscopy. Journal of Food Science, 62 (4), 734–736.

Quality Evaluation of Apples Vincent Leemans and Olivier Kleynen Gembloux Agricultural University, Department of Mechanical Engineering, Passage des Déportés 2, B-5030, Gembloux, Belgium

1 Introduction The apple is a fruit that is produced and consumed world wide. Its production is rated at over 60 × 109 kg in 2005, with the most important producers being the People’s Republic of China (25 × 109 kg), the European community (25 countries, 7.5 × 109 kg), the United States of America (4.25 × 109 kg), Turkey (2.55 × 109 kg) and Iran (2.4 × 109 kg). The number of cultivars is estimated to be over 7500, but only a few of these are subject to mass production and appear on supermarket shelves. The quality of apples is strictly regulated, and they are classified into categories Extra, I, and II by standards established by international organizations such as the OECD (International Standard on Fruits and Vegetables – Apples and Pears, 1970) (The category names may vary between countries; a category III theoretically exists but, to the knowledge of the authors, is not used.) The fruits not complying with the minimal requirements of the lowest class are excluded from the fresh market and used by the food industry (stewed apples, juice, or cider) or for animal feeding (the cull). The quality encompasses different aspects, the most important of which concerns the presence of defects and the size tolerated within each class. The shape of the fruit is also expressed in those standards. National and distribution “standards” usually specify size, grade and color classes. The quality of the fruits presented to the fresh market has a major influence on their price. The distributors demand batches of homogeneous quality, while the intrinsic quality of these biological products varies widely, from fruit to fruit, from one orchard to another, and in time. The grading is thus an essential step; however, it is a tedious job, and it is difficult for the graders to maintain constant vigilance. If this task could be performed by machine vision, the results would be more objective; it would also save labor and enhance output. This chapter presents recent developments in this domain. The grading of an apple by using computer vision begins by acquiring an image and finishes with evaluation of the fruit’s quality. Meanwhile, the information contained in Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


9

214 Quality Evaluation of Apples

Image acquisition

Fruit localization

Shape evaluation

Color measurement

Calyx & stalk-end localization

Image segmentation

Object recognition/ defects identification

Apple grading Figure 9.1

Diagram showing the path of the information from image acquisition to evaluation of quality.

the image(s) is processed following, more or less, the diagram presented in Figure 9.1. Not every step is encountered in every study, but this seems to be a reasonable guideline, and the organization of this chapter follows this scheme. The first step consists of acquiring the images, and this is briefly described in section 2. The first treatment consists of localization of the fruit in the image, and the determination of its boundary. Boundary analysis may be used to parameterize the shape information, which can be fed directly to the fruit quality classifier; this is discussed in section 3. The boundary is also used to determine the shape of the region of interest (ROI), including the pixels to be taken into account for subsequent procedures. The color of the fruit is then proposed for color grading and for detection of defects (sections 4 and 5). Two poles, i.e. the calyx and the stalk-end (or stem-end), are presented in apples as showing an aspect quite different from the rest of the fruit. They are usually darker areas, including the pixels that are often classified as identifying a defect by most of the segmentation algorithms. Their identification is necessary, and dedicated algorithms are often used (section 6). The segmentation results are used to grade the fruit, at a low level, with a minimal treatment, or after the different objects have been characterized by features such as their shape or their color in order to recognize the defects (section 7). Finally, the fruit’s color, its shape, and the presence of defects, their nature, and their area, contribute to the quality assessment. The quality of apples may also include other aspects related to “internal” properties such as the chemical composition (e.g. sugar content and acidity), physical

Material 215

characteristics (hardness, juiciness, mealiness) and internal defects. Though color may somehow be related to the maturity and thereby to the above properties, accurate evaluation requires other techniques (such as near infra-red spectroscopy) which will not be discussed in this chapter. Internal breakdown (such as brownheart) is not visible from outside the fruit and is thus out of the scope of this chapter, while defects such as bruising and bitter pit which are visible through the skin will be considered.

2 Material The most immediate task of an apple-grading machine is transporting the fruit. Indeed, as apples are fragile, it is a challenge to ensure that the task can be carried out at rates of up to 10 fruits per second while presenting all facets to a camera under adequately controlled lighting. In a grading line, a distance of about 0.11 m between the centers of two fruits seems to be the minimum. In other words, the fruits should be carried at a speed of about 1.1 m/s. In order to avoid blurred images, the integration time should not exceed 0.25 ms. Furthermore, the lighting should be powerful enough (around 80 W of lighting tubes per line) to be adapted to the chosen spectral bands and with adequate repartition.

2.1 Manipulation and presentation of the fruits The apples are near-spherical objects, and thus their surface cannot be presented on a plane. Consequently, there is no theoretical way to assemble different images of an apple to represent its whole surface without distortions and compromises. Figure 9.2 shows some of the possibilities, while Table 9.1 gives a summary regarding representing the apple surface. To ensure that the whole surface of the apple might be visible, several devices are used. The earliest but still most commonly used method is to place the apples on bi-conical rollers so that they evolve under the camera while rotating. With the apple being placed on rollers and moved perpendicularly to the optical axis of the camera, about two-thirds of the surface is visible; this may be enough to estimate its ground color and the blush area ratio, but not for defect detection. The rotational poles cannot be seen from above, and thus mirrors are added to the side of the sorting line. By assembling successive images from a matrix camera, it is possible to obtain a nearcylindrical projection of the surface. The fruit is placed on “rollers” that have a given angular speed. If the fruit does not slip on the rollers, the tangential speed at the contact points is the same and the angular speed of the fruit depends directly on its diameter as well as the dimensions of the ROI to be taken into account. Because of the lack of stability at a high rotational speed, this method is limited to a grading rate of around three apples per second. In a similar but more complex system, a kind of cup and small wheels, as designed by Throop et al. (2005), orientates the stalk–calyx axis vertically during its transport. The fruit is then tipped by 45◦ on to rollers and presented perpendicularly to the optical axis of a camera. A rectangular ROI is used, and a single


Figure 9.2

Apple-image acquisition diagram.

image is reconstructed (the calyx and stalk poles are then ignored). The number and the width of the ROIs are function of the diameter. In another device, two cameras inspect the line(s) with their optical axis at an angle of around 45◦ to the vertical (Leemans, 1999). If only one line is inspected, the distance from the line to both cameras is equivalent and thus all the apples are viewed at the same scale. The apple is modeled as a sphere rotating without slipping on the rollers. Two ROIs are considered. The shape of the smaller ROI is computed as the projection on the camera sensor (the charge-coupled device, CCD) of a spherical triangle delimiting the portion of the fruit assigned to each image. One apex of this triangle is at the rotational pole and the two others are at the “equator.” Their positions are determined taking into account the diameter of the fruit. The larger ROI surrounds the triangle by at least five pixels. All the pixels in this area are classified as defects or healthy tissue. On each

Material 217

Table 9.1 Main devices proposed to present the whole surface of the fruit to the camera. Support

Optical device

Surface ratio observed (%)

Rollers

One single camera

Rollers

One camera + mirrors

100

Near cylindrical projection + rotational poles views

Rollers

Two cameras

100

Near bi-conical projection

Robot arm

One camera

80

66

Remarks

Near cylindrical projection

Tetrahedral projection

view, every object (defect, calyx, stalk-end) is characterized by a number of features, including the position of its center of gravity. In order to evaluate the quality correctly, each defect has to be counted once and once only, although it may appear on several images. To solve this, the defects with their center of gravity within the “triangle” are considered. If the same defect appears in another image, its center of gravity should be outside the corresponding triangle. The apples are then graded according to the entire set of attributes of all the retained defect. These devices share the same drawback in that the assumption is made that the apples spin without slipping or tilting (i.e. the rotational axis remains the same during one turn). To overcome this, Moltó et al. (1998) manipulated the fruit by two robot arms but at a low rate of about one fruit per second. In the study by Guedalia (1997), apples were conveyed on ropes while images were acquired by three cameras; however, a small part of the apple surface was blocked by the ropes. The geometrical relationship between the different images of the fruit is not obvious, and thus many researchers work on separate images. The blush area ratio of a fruit is computed using the whole set of views. For the defects, there are various possibilities – for example, to evaluate the defects in each view and grade each view individually, the rating of the fruit being the one given by the worst view; or to evaluate the defects in each view and compute global parameters such as the total area of defects or the bigger defect. The support for the fruit constitutes the surrounding area of the fruit in the image, and obviously it should be of relatively high contrast to the fruit. Figure 9.3 shows a bi-color apple placed on two different backgrounds, one bright and one dark. The contrast is sufficient in the red channel for the dark background (and also in the NIR wavelength bands, unshown), and in the blue channel for a bright background (or both,


Red

Green

blue

Figure 9.3 Bi-color apple (green ground color left, red blush right) placed on a part white and part black background. From left to right, these are red, green, and blue channels of a RGB color image.

using a blue background, for example). A dark background seems to be used most often, but bright blue and white can be encountered. A bright background may present shadows and is more subject to unevenness. When the fruit is well contrasted against the background, fruit localization is undertaken by classical supervised or unsupervised threshold techniques.

2.2 Lighting The aim of the lighting system is to provide irradiance that provides the most relevant information about apple quality after being reflected. Two major concepts should be considered: repartition (i.e. its geometry) and spectral content. 2.2.1 Lighting geometry The apple surface presents different degrees of glossiness, depending on the variety and the maturity. Specular reflection seems unavoidable, but diffuse lighting can minimize its effects. The geometry of the lighting should make the image of the fruit either as uniform as possible (provided that its reflectance is uniform), or give it known variations. In an attempt to fulfill the former requirement, half-spherical lighting chambers are designed. The fruit is placed at the center of the chamber, and the light sources are placed below the fruit and illuminate the inner surface of the chamber, which is painted flat white to provide a diffuse and uniform light (Moltó et al., 1998). This device is used with a robot arm manipulator, but in practice it is not suitable for roller sorting machines. A cylindrical lighting tunnel is therefore built, based on the same principles, allowing the fruit to pass through it. Figure 9.4 illustrates some of the designs. In such devices (Miller and Drouillard, 1997; Leemans, 1999; Throop et al., 2005) uniform lighting is possible at the direction perpendicular to the traveling direction, but is difficult to achieve in the direction of travel because the apples are quite close to one another. In some cases additional lighting devices are added to the extremity of the lighting chamber. For other devices, only part of the image can be used; however, as the fruit is rotating, there are ways to observe the whole surface under the correct

Material 219

(a)

(b)

(c) Figure 9.4 Different image-acquisition designs. (a) The fruit is placed on a conveyor system (here a belt is schematized) and illuminated by the diffuse reflection of the light provided by lamps (here lighting tubes and bulbs) placed beneath the level of the fruit. The camera observes the apple from above through a hole in the reflector. (b) Cross-section of a lighting tunnel where the fruit is placed on rollers and observed by two cameras through the reflector. (c) A view of a two grading-line prototype based on the former concept.


conditions. Lighting forming a horizontal layer above the apples was used by Wen and Tao (1998); this had the advantage of covering several lines, but the drawback of presenting strong illuminant variations from the center of the fruit to its border. 2.2.2 Spectral composition The spectral composition of the incident light depends mainly on the lighting sources, and cannot be easily tuned. Fluorescent lighting tubes are mainly used for image acquisition in the visible part of the spectrum, while incandescent bulb lamps are generally used for inspection in the NIR part. Some researchers combine both to extract spectral information at different wavelength bands to enhance the defect detection (Kleynen et al., 2004), while others use this method to extract two different kinds of information at the same time. Yang (1993) used the visible spectrum for defect localization and the NIR region for fruit curvature analysis, while Penman (2001) used the green to NIR part of the spectrum for defect localization and the blue region for curvature analysis. Light-emitting diodes have also been used recently (Throop et al., 2005). These present the advantage of emitting a narrow bandwidth.

2.3 Image acquisition devices The spectral sensitivity of the image acquisition devices and the number of “channels” acquired depend on the development of technology. As a guide, in the 1980s and earlier, monochrome cameras were used; in the 1990s, color cameras were considered. More recently, Mehl et al. (2004) have used a hyperspectral imaging system to detect apple surface defects. Since this imaging technique provides a large amount of data, which it takes a great deal of time to acquire and to process, it cannot be transferred to an industrial machine. Taking into account practical considerations, Kleynen et al. (2003, 2004) selected four wavelength bands in the visible and NIR spectra and developed a four-band multispectral vision system dedicated to defect detection on apples. The system had the potential for industrial application. Mid-infra-red cameras were also employed in order to recognize the stalk-ends and calyxes (Cheng et al., 2003), but their high price prevents their use in commercial grading machines for the moment.

2.4 The image database The grading machines rely on a model of an ideal fruit, and data from the observed apple are compared with those of the model. This model is built thanks to a database. However, the important question is how many fruits should be considered while building such a database. This depends, of course, on the variability of the parameters. The quantities given here may be considered a general guideline. Regarding color, it is most important to have fruits representative of the color variability in space and time (the color changes according to the picking date and time of storage). A hundred fruits, being representative of the variability at a particular moment (and thus including the extremes), and four samplings a year (thus 400 apples) seems proper.

Material 221

Regarding shape, a variety presenting a well-defined form (such as Golden Delicious) is easily measured, and a hundred apples would be sufficient. For varieties with a more variable shape, the number of samples should be increased accordingly. For defects, the simple answer is, as many as possible. Since variability of the blemishes is extremely large (see Figure 9.5), their detection and the fruit grading usually require advanced algorithms and the estimation of many parameters. The ideal system should be able to expand the database in time. At the very least, several hundred apples should be considered; a thousand or even more is preferable. Since one year is different from another, this database should be built across several years. A particular blemish may represent an important proportion of defects for one year or for one location, but might not be encountered for the several years afterwards. It is thus important to vary the origin of the apples with regard to space and time, to take into account the “inter-orchard,” “in-year,” and “inter-year” variability.

(a)

(b)

(c)

(d) Figure 9.5 Different kinds of defects, showing the variability in color, size, and texture. Left to right: (a) fungal attack, old mechanical damage, recent bruise (within the dotted line), old bruise; (b) russet, attack by apple fruitminer (Marmara Pomonella), bitterpit, old scar; (c) reticular russet, reticular russet, aphid attack (Disaphis plantaginea, leaving no “patch’’ but a textured surface), frost damage; (d) four healthy fruits.


3 Shape grading The shape of the fruit is defined in the standards in words such as “typical of the variety” for class Extra, “showing slight misshapeness” for class I, “showing strong misshapeness” for class II, or to be rejected. For each variety, the typical shape is again expressed as spherical, elongated, conical, flattened, with marked ribs, and so on, which are impractical definitions for image analysis. Most researchers ask experts to grade fruits into shape classes and to find suitable shape parameters. Different kinds have been used, ranging from shape indexes such as circularity, eccentricity, and Hu’s invariant moments (Hu, 1962) to fractals or Fourier descriptors. In the latter, the distance from the center of gravity of the fruit to its boundary is expressed as a function of the angle from the horizontal (or any other reference). The amplitudes of the first few harmonics (computed using a fast Fourier transform) can be used to grade Golden Delicious apples with an error rate of 6 percent using a linear discriminant analysis (Leemans, 1999). Other varieties such as Jonagold, which is a cross between a rather elongated variety (Golden) and a flat one (the Jonatan), present highly variable shapes, and can show the shape of either of their ancestors. In this case a “misshapen” shape owing to a pollination problem might be more complicated to detect. The main drawback is that the fruit have to be presented to the camera with their stalk–calyx axis perpendicular to the optical axis, which requires a mechanism such as the one proposed by Throop et al. (2005). However, a failed orientation rate of 2.3 percent occurs.

4 Color grading Apples usually present two colors, i.e. the ground color varying from green to yellow, and the degree of ripeness and the blush varying from pink to deep red. Many varieties, such as Boskoop, Braeburn, Gala and Jonagold, present both colors, while others present mainly one – for example, Granny Smiths are normally light green, Gingergold and Transparent are whitish green, Golden Delicious are green to yellow-green but may show a slight pinkish blush, Fuji are generally red, and Red Delicious are deep red. The color criteria given by international standards, such as the European Community (EC) no. 1799/2001, are often complemented by national or auction standards. It should also be noted that many varieties of apples also present varietal russet, which will be discussed in the next section. Early studies concerning apple color (Lespinasse, 1971) are at the root of the picking color charts, using the color space available at that time. The relationships between the ground color at harvest and colors during storage were studied (at that time ultra-low oxygen storage facilities were not common and the fruit matured much more quickly during storage than is the case nowadays). As a result, the picking date could be chosen taking into account the ground color and the expected storage duration. Others (Ferré et al., 1987; Shrevens and Raeymakers, 1992) studied the relationship between the L*a*b* space and the maturity or the ground color standards. It should be noted that

Color grading 223

250

250

250

250

200

200

200

200

150

150

G

150

150

G

G

G

100

100

100

100

50

50

50

50

0 50

100

150 R

200

250

0

0

0 0

0

50

100

150 R

200

250

0

50

100

150

200

250

0

R

50

100

150

200

250

R

Figure 9.6 Relative frequency diagrams, projection on the plane determined by the R and G axes. Distribution computed on 100 fruits of different maturity classes, from the least (left) to the most (right) ripe.

the color spaces used for human-based color assessment (such as the L*a*b*) are not intrinsically the most suitable for computer grading. Figure 9.6 shows the relative frequency distributions for the luminance of the red channel vs the green channel for pixels of bi-color Jonagold apples of different ripeness levels. Images used were acquired with a three-CCD color camera. The ground color (shown in the upper right of each of the diagrams in Figure 9.6) varies with the maturity, while the blush (bottom left) does not. The color picking and grading charts are representative of two facts: apples presenting an important ground color are graded according to their color into classes from green (associated with freshness and chosen by people who prefer acidic fruits) to yellow (associated with maturity and sweetness); and for apples showing a distinct blush the proportion of blush area is important. From the image-analysis point of view, this means that the pixels belonging to the ground color should first be separated from those composing the blush area. As it can be seen in the frequency-distribution diagram in Figure 9.7, the frequencies between the two modes corresponding to the blush and the ground color are quite low. This suggests that the transition (the pigment change) appears to be quite fast. Because of the non-Gaussian distribution of both colors, the pixels are best classified using neural networks into either ground color and blush (Leemans, 1999) or different color classes (“normal red,” “poor color red,” “vine,” “upper and lower background color”) and injured (Nakano 1997). Evaluation of the proportion of the blush area is straightforward. The attribution of a ground color class for the fruit is based on the mean or, better, on the median ground color, since the latter is less influenced by the asymmetry of the distribution. Figure 9.7 shows scatter diagrams, in the green–red and blue–red planes, of the median color of 80 Golden Delicious apples graded by an auction expert into four ground-color classes. The dispersion of the median points is similar for each class, while the mean of the distribution is close to a straight line. The first canonical variate can be used to discriminate the medians into the color classes with an error rate of 9 percent, according to the experts. (The first canonical variate maximizes the ratio between the variance within the classes and the variance between the classes. It is given by the first eingen vector of the matrix A = FE−1 , where F is the factorial sum of the products of deviates matrix, and E is the residual sum of products of deviates matrix.) It can be seen from Figure 9.7 that part of the error may be attributed to the experts. The hue parameter h


220 210 200 G

190 180 170 160 150 150

170

190

210

230

250

100 95 90 85 80 B 75 70 65 60 55 50 150

170

R (a)

190

210

230

250

R (b)

100 B

50 0 100

150 R 200 250 200

250

G

150 (c)

Figure 9.7 Scatter diagrams of the median color of 80 Golden Delicious apples, graded by an expert into four color classes, from the greenest to the yellowest: = A++; ♦ = A+; = A; x = Ar. (a) Red (R) and green (G) plane; (b) red (R) and blue (B) plane; (c) the red (R), green(G), blue (B) space perpendicularly to the first canonical variate (the discriminant functions are visible).

is also very effective, although, being a non-linear combination of the red, green, and blue values, it requires more computations. A RGB image contains all the information necessary to grade fruits according to their color. When a dedicated wavelength imaging device is used for apple defect recognition (Kleynen et al., 2004), the selected wavelengths are primarily chosen to enhance the defect detection. These wavelengths are not well suited for ground color vs blush segmentation, and a supplementary wavelength band located in the green visible spectrum (500–600 nm) should be used. Indeed, as illustrated in Figure 9.8, in that wavelength band the reflectance differences between the ground color and the blush are highest.

5 Evaluation of surface defects External defects have many different origins, including fungal attack, insect or bird bites, various mechanical wounds, and physiological factors such as frost damage and sunburn. As presented in Figure 9.5, these are expressed by variable colors, textures, boundaries (frank and diffuse), shapes (circular and irregular), and dimensions.

Evaluation of surface defects 225

100 90

Ground color Blush

Normalized reflectance (%)

80 70 60 50 40 30 20 10 0 400

500

600

700

800

900

1000

1100

Wavelength (nm) Figure 9.8 Spectral reflectance of the ground color and the blush of Jonagold apples.

Furthermore, healthy tissue also has its own variability and texture. Each fruit presents two areas – the calyx and the stalk-end – which are not defects but may present similar aspects. Russet is produced by the fruit itself and is not regarded as a defect as long as its size and repartition is “typical of the variety.” This complicates defect recognition and proscribes the use of simple methods such as the measurement of global parameters for the whole area of the fruit, as presented (amongst others) by Heineman et al. (1995). Defects can be observed because of their different luminance compared with the surrounding sound tissue. Yang (1994) described the aspect of a mono-color apple and its defects as they might be seen in a monochrome image. The fruit appeared light green, with the mean luminance depending on the fruit color. Apples presented lenticels, creating small variations comparable to noise. It was also noted that the reflection factor decreased from the center to the boundary. The defects were usually darker than the healthy tissue, but their contrasts, sizes, and shapes might vary strongly. For these reasons, the author assumed that simple techniques such as “thresholding” or background subtraction gave poor results. Consequently, researchers pretreated the images by removing the outer parts, which were observed under an unfavorable angle (Leemans, 1999; Unay and Gosselin, 2005). It was also considered beneficial to compensate the non-uniformities algorithmically with a flat-field correction by computing a correction coefficient function according to their distance to the center of the fruit (Wen and Tao, 1998), or with background correction by a flat white spherical object of equivalent size (Throop et al., 2005). The images were then segmented by applying a threshold, set empirically or algorithmically (Ridler and Calvard, 1978; Otsu, 1979; Kapur et al., 1985). Yang and Marchant (1996) presented a method based on a topological algorithm (called flooding) followed by a snake algorithm for the detection of “patch like defects” which did not require the flat-field correction.


It is unlikely that these methods would work on monochrome images of bi-color apples acquired in the visible part of the spectrum up to 650 nm, because the variation in the reflectance between the ground color and the blush is far too important (Figures 9.3, 9.8). However, they remain valuable for monochrome images acquired in the NIR wavelength bands or for mono-color green fruits. In color and multispectral imaging, defect detection can be carried out in several ways. The different algorithms applied to process both the kinds of image and the data issued from these may be similar. The term “color” (in quotation marks) will hereafter be used for both color or multispectral images. In multispectral imaging, detection may be performed separately for each wavelength band and the data may be fused afterwards (Throop et al., 2005). More efficient methods take into account the simultaneous variations of the different spectral components. Working on Golden Delicious apples (mono-color fruits), Leemans (1999) evaluated the difference between the color of each pixel and the average of the fruit by the Mahalanobis distance dM 2 : dM2 = (x − x)−1 (x − x) with x being the color vector [r, g, b] of the pixel, x the mean color vector of the fruit, and the covariance matrix of the color. This is in fact the generalization of a confidence interval. When the distance is lower than a threshold, the corresponding pixel is considered as healthy tissue; otherwise, it is assigned to a defect. Samples of segmentation results are presented in Figure 9.9. Slight under-segmentation may

(a)

(b)

(c)

(d)

Figure 9.9 Examples of defects on Golden Delicious apples (top) and segmented images using (middle) the Mahalanobis distance and (bottom) after the second step. Defects origin: (a) russet resulting from an insect bite; (b) scab; (c) diffuse russetting; (d) bruising.


be observed for a low-contrast defect (the russet) while a part of the boundary is erroneously segmented as defect, which is not a problem because it is out of the ROI. This kind of algorithm has the advantage of being unsupervised. The dispersion parameters of the color distribution have to be known before segmentation, but they can be measured once, off-line, on healthy fruits that are selected to be representative of fruit color. Moreover, since each pixel color is compared to the mean color, if small disturbances occur – for example in the illuminant changing both the mean and each pixel values – the distances are not much affected and the algorithm remains robust. Nevertheless, it works only if the probability density function (PDF) of the fruit color is, at least approximately, a Gaussian distribution, which is the case for mono-color fruit such as Golden Delicious. For bi-color fruits in the RGB space, this assumption is far from being fulfilled. As can be observed for Jonagold apples in Figures 9.6 and 9.10, these distributions are multimodal. The different modes correspond to the ground color and the blush for the healthy tissue, and the different origin of the defects. Moreover, the distributions are close to each other. However, discrimination between the defects and the 250

250

200

200

150

150

G

G 100

100

50

50

0

0 0

50

100

150

200

250

0

50

100

R

150

200

250

150

200

250

B

250

250

200

200

150

150

G

G 100

100

50

50

0

0 0

50

100

150 R

200

250

0

50

100 B

Figure 9.10 Relative frequency diagrams of healthy Jonagold apples (top) and of defects (bottom). Left: projection on the plane determined by the R and G axes; right: projection on the plane determined by the B and G axes.


(a)

(b)

(c)

(d)

(e)

Figure 9.11 Sample images of Jonagold (top): (a) ripe, healthy fruit; (b) healthy fruit; (c) poorly contrasted rotten fruit; (d) russet; (e) scab. The second row gives the a posteriori classification probabilities (high probability of healthy tissue is shown as white). The third row shows results of the segmentation after the second step; the background is black, the blush is dark gray, the ground color is light gray, and the defects are in white.

healthy tissue is possible using the a posteriori classification probabilities computed by Bayes’ theorem. It is necessary to estimate the PDFs of the color of the healthy tissue and the defects. Taking into account the complexity of the distributions, Leemans (1999) proposed a numerical model. In this case, defects had to be previously marked on images by an operator to obtain their color frequency distribution. In order to segment the images on-line, the PDFs were estimated using the kernel method and the probability that a pixel of a given color belonged to the healthy fruit or to a defect was computed off-line and stored in a table. The model was compared regarding color coded on six bits and seven bits per channel. Similar results were experienced, and the former was consequently chosen to reduce the size of the table. Figure 9.11 presents the a posteriori healthy tissue classification probabilities of four sample images (high probability of healthy tissue is shown in white). In order to segment defects on San Fuji apples, Nakano (1997) used a backpropagated neural network with two layers to classify pixels into six color classes by pixel features including position and the mean color (in RGB). Five of the classes were representative of the colors of healthy tissue, while the other was for defects. The same kind of neural network was used by Unay and Gosselin (2005) on four wavelength-band multispectral images of Jonagold apples acquired with the imaging device developed by Kleynen et al. (2004). Both methods (Bayes’ theorem and back-propagated neural networks) need preliminary supervised classification of the pixels, which makes them sensitive to a change in the illuminant. To solve this major drawback, Kleynen and Destain (2004) proposed an unsupervised defect segmentation method to process multispectral images of Jonagold


(a)

(b)

Figure 9.12 Result of the unsupervised segmentation of multi-spectral images of defects (ringed) which are typically poorly segmented with standard color cameras and supervised segmentation. (a) Hail damage without skin perforation; (b) scald. Top: green visible spectral band (centered on 500 nm); bottom: result of segmentation (dark = defective tissue, white = healthy tissue).

apples. This method did not depend on parameters previously computed on sample images, and was based on the analysis of the probability density distribution of the spectral components of the image. The modes and the valleys of the distribution were detected by a hill-climbing method using a density gradient estimate derived from the “mean shift” procedure (Comaniciu and Meer, 2002), whose variations were correlated to local maxima of the PDF. This procedure leads to a variable number of clusters. In order to obtain only two tissue classes (defect and healthy tissue), the Bhattacharyya distance (generalization of the Mahalanobis distance to populations with covariance matrix not supposed equal) was used to identify the two most distant clusters of the distribution. Starting from these two seed clusters, the probability density distribution was then divided into two main clusters by regrouping the other clusters according to the nearest neighbor method. Figure 9.12 presents the segmentation results regarding two kinds of defects, which are generally poorly segmented with supervised methods and classical color-imaging devices. When the image has been segmented, several researchers have considered that refinements might be possible. Yang and Marchant (1996) used the snake algorithm, an active contour model. The limits of the objects were modeled as a string attached to the one initially segmented position by a spring, attracted by the dark area and presenting


a certain rigidity (inducing a bending moment). The boundary was reshaped by minimizing the total energy of the system. Three parameters were fitted: the weight, the spring, and the boundary rigidities. This caused initial over-segmentation, which was usually the case with the flooding algorithm. Leemans (1999) considered, for monocolor fruits, a second segmentation step. After the first step, the mean colors of the defects and of the healthy tissue were computed, and, for each pixel, the distances to each mean color were computed. The pixel was reassigned as healthy tissue or as a defect according to the closest mean. The examples given in Figure 9.9 show the segmentation enhancement of lower-contrast defects. For bi-color apples, researchers proceeded in the similar way but in a local area (Figure 9.11). Wavelengths in the red and NIR parts of the spectrum are mostly encountered for defect segmentation. As can be observed in Figure 9.7, the reflectance in the blue part is low (0.1) and it is highly variable in the green and yellow part. However, as demonstrated by Kleynen et al. (2003) while testing the whole set of three or four wavelength bands, these parts of the spectra also contain valuable information, because the corresponding standard deviations are also low.

6 Calyx and stalk-end recognition The calyxes and stalk-ends are “defect-like” objects, and are usually spotted by classical defect segmentation algorithms. Consequently, these have to be recognized, either before or after segmentation. The calyxes and stalk-ends present an aspect far less variable than defects, even though many factors may influence it. The russet in the stalk-end and around it is often a varietal characteristic, and as such should not be considered as defect unless it is overgrown. The stalk-end and calyx may be positioned centrally on the fruit image, or at its periphery. Nevertheless, they remain circular objects that are dark in the centre and have fuzzy boundaries. In order to locate these cavities, the pattern-matching method is a simple and useful method. The principle is to match a known image or a pattern (Figure 9.13) with

(a) Figure 9.13

(a) Stem-end and (b) calyx patterns.

(b)

Defect recognition and fruit classification 231

another by computing cross-correlation and finding the maximum. To compensate the sensitivity for a given model, a mean image computed from five stalk-end images was used by Leemans (1999). The author, working on RGB images, also showed that the green and the red channels gave similar results for mono-color fruits such as Golden Delicious and for bi-color fruits such as Jonagold. When the maximum value of the correlation coefficient was used to distinguish defects having a similar aspect (mainly circular defects), the error rate was 3 percent. The calyxes and the stalk-ends were well recognized, but some defects (such as circular defects and misshapenness owing to insect bites) were misclassified. Yang (1993) and Penman (2001) both used structured lighting in the NIR or in the blue spectral bands to reveal the different curvature of the fruit around the cavities, and detected the defects in another part of the spectra. Cheng et al. (2003) showed that a couple of NIR/MIR cameras were useful in revealing the calyxes and stalk-ends. Unfortunately, the high cost of such equipment is prohibitive. Unay and Gosselin (2007) proposed a technique based on the segmentation of multispectral images with one channel (750 nm) and object classification. More than 35 parameters regarding “color” (in each of the four used channels), texture, and shape were extracted from each object. After selection of the most relevant parameters and the most discriminant method, the authors showed that just nine parameters were enough, and that the support vector machine gave the best result (using k-fold cross-validation) with an error rate near zero for the calyxes and stalk-ends and of around 13 percent for defects. Guedalia (1997) employed a set of parameters measured for each object to determine whether the object was a calyx, a stalk-end, or a defect. When the cavities have been located, some researchers simply remove the corresponding pixels from the apple surface while others process them during defect recognition (discussed in the next section). Figure 9.14 presents the results of the flood-filling method used by Kleynen and Destain (2004) for segmenting the calyxes and stalk-ends on the basis of a seed pixel corresponding to the maximum value of the cross-correlation.

7 Defect recognition and fruit classification Once the image has been segmented, information is extracted in order to grade the fruit. The size, “color,” shape, and texture of the object, as well as the distance from center of gravity of the object to the calyx or to the stalk-end, may be evaluated for each object. The number of objects detected in the segmented image may vary from none in the ideal case of a healthy fruit correctly segmented, to 100 for some kinds of russet. As classifiers require a fixed number of input parameters, this information has to be summarized. The different approaches consist of extracting global statistical parameters on the whole set of pixels, characterizing each object, and grading the fruit on the worst one. The latter two can also be referred to as recognizing the defect individually and grading the fruit according to the standards of examples.


Figure 9.14 Samples of results of calyx/stem-end segmentation by a flood filling algorithm. The center of the white cross is the seed pixel of the algorithm and the white contour line is the boundary of the filled area.

For most of these methods, the grading is based on the information coming from one image. As several images are required to observe a whole fruit, we can suppose that each image is graded separately, and the grade given to the whole fruit is the lowest found.

7.1 Features The most evident and commonly used size and shape parameter is the area. It may be computed from each object or from the whole fruit by the sum of the effective pixels. In the latter case, it can be used directly (or as defect area ratio, i.e. the ratio of the total defect area to the fruit area) to grade the apple. The distance from the center of gravity of the object to the center of gravity of the fruit is also used as global or object feature. The perimeter, the major inertia moment, and the ratio of the inertia moments are also used to evaluate the shape of defects individually. The most encountered “color” parameters are the mean value of each channel, or a distance from the mean “color” of the object to the mean “color” of the fruit – i.e. its contrast. This latter distance may be computed for each channel (one parameter per channel, usually the absolute differences) or in the color space (i.e. one parameter, the Euclidian or the Mahalanobis distances). The texture may be evaluated by the standard deviation in each color channel and by the mean and standard deviation of the image gradient for a particular channel.

Defect recognition and fruit classification 233

Invariant moments computed on the co-occurrence matrix are also used, although a greater computational load is experienced. A step-wise process with the error classification rate used as a criterion is usually proposed for parameter selection. Normally 12 to 15 parameters are retained, representing the different categories (shape, color, and texture).

7.2 Global statistical parameters Some parameters are extracted directly at pixel level: the total area; the defect area ratio; and the mean, the median, and the standard deviation values of each spectral channel. Several researchers have considered the area of the largest defect. In most cases, each image was processed separately and the fruit was graded according to the worst case. Throop et al. (2005) used the total area of the defect in an image representing two-thirds of the fruit surface. The apples were graded according to the USDA standards, with an error rate of 12 percent. The fruit being mechanically oriented, the calyx and stalk-end were, however, not inspected. In order to grade Jonagold apples by multispectral images, Kleynen et al. (2004) employed the mean, the median, and the standard deviation values of the 450-, 750-, and 800-nm spectral components plus the defect area ratio. The authors achieved an error rate of 11 percent with linear discriminant analysis. The calyx and stalk areas were detected and segmented prior to defect detection, and the corresponding area were ignored. Unay and Gosselin (2005) proposed a similar set of parameters, and obtained similar results by using a support vector machine. Another technique developed by Guedalia (1997) is to perform a principal component analysis on the whole object feature set before using a supervised grading (error rate of 33 percent).

7.3 Hierarchical grading based on object supervised classification The basic idea is to recognize a defect’s origin by means of supervised defect classification. The standard separates the defects into flesh defects (unacceptable, whatever size) and skin defects (which degrade the fruit according to their size as presented in Table 9.2). It should be noted that bruises are flesh defects, and any fruit presenting a bruise should be rejected. The steps to achieve fruit grading are: 1. Compute shape, color, and texture features of each object in the image 2. Classify the object into one defect category 3. Grade the fruit according to the standards. This procedure is well suited to blobs or patch-like defects, but, as can be observed in Figure 9.8, the reality is more complex. Some defects present a more scattered aspect, such as the diffuse russet, while others (bruises and russet, mainly) have a color very close to that of the healthy tissue. Scattered or reticular russet is often segmented as


Table 9.2 Maximal dimensions for defects accepted in each category, according to OECD. Class

Extra

Defect

Area (mm2 )

Length (m)

Area (mm2 )

Length (m)

Area (mm2 )

Length (m)

Scab Minor defects Russet

0 0 0

0 0 0

0.05. An improved version of an ellipse-fitting algorithm combined with the mathematical morphology method was developed and tested (Zhang et al., 2005). Typical touching kernel patterns of four grain types, namely barley, CWAD wheat, Canada Western Red Spring (CWRS) wheat, and oats, obtained from composite samples from several growing locations across the western Canadian prairies were used to test this algorithm. The accuracies of separation were 92.4 percent (barley), 96.1 percent (CWAD wheat), 94.8 percent (oats), and 97.3 percent (CWRS wheat). A morphological image-processing algorithm based on watershed segmentation of a distance transform graph of connected binary imagery was developed by Wang and Paliwal (2006). The algorithm dealt with an “oversegmentation” problem in original watershed segmentation by reconstructing internal markers through a series of morphological operations. The internal markers were then used to join overly segmented parts belonging to the same component. Closed boundaries of each connected component were finally pruned and extracted. The algorithm was applied to separate touching kernels of six grain types, namely CWRS wheat, Canada Western Hard White (CWHW) wheat, CWAD wheat, six-row barley, rye, and oats. The segmentation method was most successful on the three types of wheat kernels, and achieved correct segmentation rates of 94.4 percent (CWRS wheat), 92.0 percent (CWHW wheat), and 88.6 percent (CWAD wheat). The method was not as suitable for the three other grain types, with segmentation rates of 55.4 percent (oats), 79.0 percent (rye), and 60.9 percent (six-row barley). Sound CWRS wheat kernels were mixed with CWAD wheat and broken wheat kernels, so that they were in contact, and were segmented using a developed watershed algorithm. Five geometric features were extracted from disconnected binary images, and linear classifiers based on Mahalanobis distance were used to identify wheat dockage. The linear classifier identified 96.7 percent of adulterated CWAD wheat kernels and 100 percent of broken CWRS wheat kernels.

2.6 Morphological, color, and textural algorithms Once images are acquired, algorithms are needed for thresholding, pre-processing operations, and segmentation, and for feature extraction from digital images of various types of cereal grains and dockage fractions. Such algorithms were developed and evaluated over several years at the CWBCGSR (Majumdar et al., 1996a, 1996b, 1999; Nair and Jayas, 1998; Luo et al., 1999a, 1999b; Majumdar and Jayas, 1999a, 1999b, 2000a, 2000b, 2000c, 2000d). Paliwal et al. (2003a) further improved these algorithms, which were coded in Microsoft Visual C++ environment. The program is used to

360 Quality Evaluation of Wheat

extract morphological, color, and textural features from different grains and dockage fractions. A variation of the same program was used to extract color and textural features from bulk samples of grain (Visen et al., 2004b). The program can batch process a large number of image files stored on local and remote computers (connected by network), and is smart enough to skip corrupted and non-existent files. The program output (i.e. features of the objects in the image) can be written to a new text file or be appended to an existing file. The output file consists of information about the specific filenames from which the features of an object were extracted. This facilitates the back-tracking of image files and their constituent objects from the corresponding feature values. The contents of the output text files were tab delimited to enable easy export to spreadsheets. The program is flexible enough to incorporate new features without necessitating any major changes in the core program. The modular nature of the program enables the user to choose specific features that must be extracted from the objects in the image files. 2.6.1 Morphological features

Morphological features illustrate the appearance of an object. Algorithms were developed to extract morphological features based on basic size features (e.g. area, perimeter, bounding rectangle, centroid, lower-order moments (normal, central, and invariant), length and width, and angle of orientation) and derived shape features (e.g. roundness, radius ratio, box ratio, area ratio, aspect ratio, and the coefficient of variation of radii). 2.6.2 Gray-scale and color features

The gray values for monochrome images and the ratio of primary colors (i.e. red, green, and blue) for color images are used for object recognition. The three primary colors (RGB) are sometimes converted into the hue, saturation, and intensity (HSI) system or the L*a*b* (CIELAB) color scheme for easy human perception. In the HSI system, hue represents the dominant wavelength (i.e. pure color), saturation refers to the amount of white light mixed with the hue or the pure color, and intensity is the brightness of the achromatic light. In the L∗ a∗ b∗ color scheme, L∗ represents the lightness of the color (L∗ = 0 yields black and L∗ = 100 indicates white), its position between magenta (positive values of a∗ ) and green (negative values of a∗ ), and its position between yellow (positive values of b∗ ) and blue (negative values of b∗ ). Algorithms were developed to extract color features, based on means, variances, ranges, histograms, and invariant moments of red, green, and blue bands. 2.6.3 Textural features The texture of an object can be described based on the spatial distribution of image intensities. Textural features thus provide information on the surface properties of the objects, such as smoothness, coarseness, fineness or granulation. For example, a smooth object has low variation in spatial intensities, whereas a coarsely textured object has highly variable spatial intensities. Textural features can be described by Fourier transformation or statistical approaches. However, sometimes two objects can have the same morphological and color features; therefore, algorithms were developed to extract textural features from gray-level histograms (GLH), gray-level co-occurrence

Machine vision 361

matrices (GLCM), and gray-level run-length matrices (GLRM) for red (R), green (G), and blue (B) bands, and different combinations of RGB bands. The gray-level co-occurrence matrix provides information about the distribution of gray-level intensities with respect to the relative position of the pixels with equal intensities. The gray-level run-length matrix represents the occurrence of collinear and consecutive pixels of the same or similar gray levels in an object. Readers are referred to Gonzalez and Woods (1992); Majumdar et al. (1996b); Majumdar and Jayas (2000a, 2000b, 2000c, 2000d); and Karunakaran et al. (2001) for details of all the morphological, color, or textural features extracted for image analysis. 2.6.4 Testing and optimization

A database was formed of high-resolution digital images of individual kernels and bulk samples of the five most common Canadian grain types (barley, CWAD wheat, CWRS wheat, oats, and rye) collected from 23 growing locations across western Canada. The constituents of dockage were also divided into five broad categories (broken wheat kernels, chaff, buckwheat, wheat-heads, and canola) and imaged. For the individual kernels, a total of 230 features (51 morphological, 123 color, and 56 textural) were extracted from these images, and classification was performed using a four-layer back-propagation network (BPN) (Jayas et al., 2000) and a statistical classifier (nonparametric). Because the shape and size information for bulk samples is irrelevant, only color and textural features were extracted for them. Different feature models, namely morphological (only for individual grain kernels and contaminants), color, textural, and a combination of these, were tested for their classification performances. The results of these classification processes were used to test the feasibility of a machine-vision based grain cleaner. For individual grain kernels, while using the BPN classifier, classification accuracies of over 98 percent were obtained for barley, CWRS wheat, oats, and rye (Paliwal et al., 2003b). Because of its misclassification with CWRS wheat, CWAD wheat gave a lower classification accuracy of 91 percent. For the dockage fractions, because of the uniqueness in their size and/or color, broken wheat kernels, buckwheat, and canola could be classified with almost 100 percent accuracy. The classification accuracies of chaff and wheat-heads were low because they did not have well-defined shapes (Paliwal et al., 2003a). The back-propagation network outperformed the non-parametric classifier in almost all instances of classification (Table 15.1). None of the three feature Table 15.1 Classification accuracies of singulated cereal grain kernels determined by the BPN and non-parametric classifier in parentheses (Visen, 2002). Grain

Barley CWAD wheat CWRS wheat Oats Rye

Classification percentages using features Morphology

Color

Texture

Top 60

Top 30

96.5 (93.2) 89.4 (90.7) 98.3 (97.0) 95.0 (91.4) 92.8 (91.4)

93.8 (71.5) 92.9 (79.1) 99.0 (92.3) 92.9 (66.8) 94.5 (87.8)

94.2 (91.4) 91.5 (92.5) 94.9 (95.6) 90.8 (91.1) 95.2 (96.2)

98.1 (90.4) 90.5 (90.3) 98.7 (97.1) 98.4 (95.8) 98.9 (98.0)

97.9 (95.1) 90.1 (90.6) 98.5 (93.6) 97.8 (94.7) 98.0 (98.2)


Table 15.2 Classification accuracies of bulk cereal grains determined by a BPN classifier (Visen, 2002). Grain

Barley CWAD wheat CWRS wheat Oats Rye

Classification percentages using features Color

Texture

All

Top 40

Top 20

98.9 100 100 98.6 99.7

100 100 100 99.6 99.9

99.9 100 100 99.8 99.9

100 100 100 99.8 100

99.8 100 100 98.3 100

sets (morphological, color, or texture) on its own was capable of giving high classification accuracies. Combination of the three improved the classification significantly. However, the use of all the features together did not give the best classification results, as a lot of the features were redundant and did not contribute much towards the classification process. A feature set consisting of top 20 morphological, color, and textural features each, gave the best results (Paliwal et al., 2003a). The better classification accuracies obtained using neural network classifiers were in accordance with earlier studies done at the CWBCGSR to compare their performance with statistical classifiers (Jayas et al., 2000; Paliwal et al., 2001; Visen et al., 2002, 2004a). To quantify the amount of impurity in a grain sample, a relationship between the morphology and mass of the kernel (or dockage particle) was investigated. An area of a particle in a given image gave the best estimate of its mass. This relationship was tested and validated for quantifying the amount of impurity in a sample before and after passing it through a lab scale cleaner (Paliwal et al., 2004b, 2005). To automate it, it is desirable that the cleaner should have a decision support system to adjust its parameters (such as vibration rate, grain flow rate, etc.) by calculating the amount of impurity being removed from the sample. This was done by calculating the change in the ranges of morphological features of the particles before and after the sample was passed through the cleaner, which was significant (Paliwal et al., 2004b). This information can be used to optimize a machine-vision based cleaner’s performance. For the bulk samples, classification accuracies of over 98 percent were obtained for all the grain types (Visen et al., 2004b). The best results were obtained using a combination of both color and textural features. Other than oats, all the grain types could be classified, with close to 100 percent classification accuracy using an optimized set of just 20 features (Table 15.2). As classification of bulk samples will be required to identify the contents of a railcar, perfect classification is not essential in such cases.

3 Soft X-ray imaging An X-ray image is formed by penetrating, high-energy photons of 0.1–100 nm wavelength passing through an object. Two types of X-ray imaging are generally practiced in the agri-food industry: soft X-rays with a wavelength of 1–100 nm, of low energy and less penetrating power; and hard X-rays (or X-ray computed tomography) with

Soft X-ray imaging 363

Detection Screen

Sample

X-ray tube

Figure 15.6 Soft X-Ray imaging system (photograph courtesy of Canadian Wheat Board Centre of Grain Storage Research, Winnipeg, MB, Canada).

a wavelength of 0.1–1 nm, of high energy and greater penetration power, which are restricted to use in high-density objects. The X-ray technique provides images based on object density differences. A soft X-ray imaging system includes a fluoroscope which produces soft X-rays and real-time images (Figure 15.6), a computer system, and a digitizer. Current X-ray systems require that kernels be placed manually on the platform between the X-ray tube and detector (Karunakaran et al., 2003a). Automation of this technology to scan a monolayer of bulk sample moving on a conveyor belt would be ideal for use in the grain industry. Real-time hard X-ray imaging systems are available for continuous food product inspection. The shielding of low-energy X-rays and development of an X-ray detector to detect soft X-rays fast enough in a continuous system are the hurdles in the development of a soft X-ray machine. However, industries are at present working towards creating such a system where these machines would be able to scan singulated grain kernels to detect insect infestation (Karunakaran et al., 2004a). These machines can scan grain kernels at the rate of 60 g/min, like a continuous machine vision system that captures color images of grain for identification (Crowe et al., 1997). X-ray images can be acquired at different voltage and current settings. For imaging grains, a 15-kV potential and 65-µA current works best (Karunakaran et al., 2003a). Images formed on the detection screen are captured by a charge-coupled device (CCD) monochrome camera and digitized into 8-bit gray-scale images at a spatial resolution unique to the system. A computer system is used for image acquisition and post-processing.

3.1 Soft X-rays for insect infestation detection in grain Artificial infestations by different life stages of Cryptolestes ferrugineus (Stephens), Tribolium castaneum (Herbst), Plodia interpunctella (Hubner), Sitophilus oryzae (L.), and Rhyzopertha dominica (F.) in CWRS wheat kernels were created. Manually separated wheat kernels (kernels placed with the crease facing down), uninfested and infested by different life stages of the insects, were X-rayed at 15 kV and 65-µA. Histogram features, histogram and shape moments, and textural features using co-occurrence and run-length matrices were extracted for each kernel from the X-ray images. A total of 57 extracted features were used to identify uninfested and infested


Table 15.3 Classification accuracies of CWRS wheat kernels uninfested and infested by stored-grain insects, using BPN and linear-function parametric classifier (all 57 features). Insect type and stages

Classification percentages using: BPN

Linear-parametric classifier

C. ferrugineus Uninfested Larvae Pupae-adults

75.7 89.6 98.8

75.3 92.8 98.0

T. castaneum Uninfested Larvae

85.7 86.6

73.3 82.1

P. interpunctella Uninfested Larvae

100 96.3

99.0 99.3

S. oryzae Uninfested Larvae Pupae-adults Insect damage kernel

99.7 98.0 100 100

99.0 98.0 99.7 100

R. dominica Uninfested Larvae Pupae-adults

100 99.8 100

98.7 99.1 100

kernels using statistical and neural network classifiers (Karunakaran et al., 2003a, 2003b, 2003c, 2004b, 2004c, 2004d). The linear-function parametric classifier and back-propagation neural network (BPNN) identified more than 84 percent of infestations caused by C. ferrugineus and T. castaneum larvae (Table 15.3). The infestations by C. ferrugineus pupae and adults were identified with more than 96 percent accuracy, and 97 percent of kernels infested by P. interpunctella larvae were identified by both the linear-function parametric classifier and BPNN. Kernels infested by different stages of S. oryzae and R. dominica larvae were identified with more than 98 percent accuracy by the linear-function parametric classifier and BPNN. The linear-function parametric classifier and BPNN performed better than the quadratic-function parametric and non-parametric classifiers for the identification of infested kernels by different insects. The soft X-ray method detected the presence of live larvae inside the infested kernels. This was achieved by image subtraction of two consecutive images of kernels that had live active insects inside them.

4 Near-infrared spectroscopy and hyperspectral imaging The near infra-red (NIR) region extends from 780 nm to 2500 nm in wavelength. The most important aspect of near-infrared spectroscopy (NIRS) as an analytical tool is that it can determine the chemical composition and physicochemical behavior of foods and

Near-infrared spectroscopy and hyperspectral imaging 365

their raw materials. This is due to the fact that NIRS analyzes the sample in a way that reflects the actual number of molecules of individual constituents in the sample (Murray and Williams, 1990). It is known that all organic matter consists of atoms, mainly carbon, oxygen, hydrogen, nitrogen, phosphorus, and sulfur, with minor amounts of other elements. These atoms combine by covalent and electrovalent bonds to form molecules (Campbell et al., 2002). Without external radiation, the molecules vibrate at their fundamental energy levels at ambient temperature. When radiated using a light source with continuous spectral output, only light at particular wavelengths is absorbed. The energy of photons at those wavelengths corresponds to the energy gaps between two fundamental energy levels, or overtones, and combinations of vibration levels. Absorption of light in the NIR region involves transfer of radiation energy into mechanical energy associated with the motion of atoms bonded together by chemical bonds (Wang, 2005).

4.1 Measurement modes of near-infrared radiation When electromagnetic radiation interacts with a sample, it may be absorbed, transmitted or reflected. Based on sample properties and forms of propagation of NIR light in the sample, measurement of this radiation can be carried out by using the following modes (Wang, 2005): 1. Transmittance. This is applied to measure transparent samples possessing a minimum light scattering effect. Usually, sample in liquid form or solvent is presented in a glass or quartz cell since NIR light is transparent to glass. The fraction of radiation (Is /Ip ) transmitted by the sample is called transmittance. In practice, transmittance is converted to absorbance as in the following relationship: Ip 1 = log (15.1) A = log T Is where A is the absorbance in absorbance units (AU), T is the transmittance (no unit), Is is the incoming light energy (J), and Ip is the transmitted light energy (J). The relationship of the concentration of a sample, the sample thickness, and the absorbance is governed by the Beer–Lambert law (Swinehart, 1972): A = abc

(15.2)

where a is a constant called the absorptivity (l/mol · per meter), b is the sample thickness (m), and c is the concentration of a sample (mol/l). 2. Transflectance. This is a modified version of transmittance. A retro-reflector is often employed behind the sample cuvette to double the optical path length through the sample. 3. Diffuse reflectance. This is applied to the measurement of solid samples and is perhaps the most accepted measurement mode in NIR spectroscopy. The Kubelka–Munk function has been introduced to describe the energy of reflected radiation using two constants called the scattering constant (s) and the absorption constant (k). For a special case of an opaque layer of infinite thickness, the


relationship could be given by the equation (Kubelka and Munk, 1931): F(R∞ ) =

ac (1 − R∞ )2 k = = 2R∞ s s

(15.3)

where F(R∞ ) is the Kubelka–Munk function (no unit), R∞ is the reflectance of the infinitely thick layer (no unit), k is the absorption constant (mm−1 ), and s is the scattering constant (mm−1 ) (Birth and Zachariah, 1976). Apparent absorbance, as given in the following equation, is used in practice instead of the Kubleka–Munk function: 1 (15.4) AR = log R∞ where AR is the apparent absorbance, also in absorbance units (AU), and is assumed to be proportional to concentration (c). 4. Interactance. This is a modified version of diffuse reflectance. The collected radiation signal travels a much longer distance in the sample, and is assumed to be richer in information on sample constituent than that collected under diffuse reflectance mode. 5. Diffuse transmittance. This is different from diffuse reflectance in that the diffuse transmittance signal is collected after light has traveled through the sample and emerged on the other side of it. This mode is often used at short NIR wavelengths with a turbid liquid or solid sample with a thickness of 10–20 mm.

4.2 NIR spectroscopy instrumentation Practical application of NIR spectroscopy has been around for several decades, and there has been a wide array of instruments available for different end-user purposes. Each kind of instrument is based on different working principles and possesses certain performance characteristics. Currently, in the agricultural and food-related fields, spectroscopic instruments can be put into two categories (Wang, 2005); dispersive and non-dispersive systems. 4.2.1 Dispersive systems

Most dispersive systems are based on diffraction gratings. According to its instrument configuration, this type of spectrometer can be divided into a scanning monochromator and a spectrograph. A scanning monochromator works by mechanically rotating the diffraction grating to tune the wavelength of light to be received by detector, whereas a spectrograph utilizes a linear array detector such as a charge-coupled device (CCD) or a photodiode array (PDA) in place of a single element detector, and light signals at multiple wavelengths can be detected simultaneously. Diffraction-grating based instruments are relatively low in cost and very capable in many industrial sectors. Drawbacks for the scanning diffraction-grating monochromators include their relatively slow scanning speed, and a degrading system performance over time due to mechanical fatigue of moving parts. Compared to a scanning monochromator, a spectrograph is faster in speed, has no moving parts, and thus is robust in structure.

Near-infrared spectroscopy and hyperspectral imaging 367

Another type of dispersive system employs electronically tunable filters, such as acousto-optical tunable filters (AOTF). Using AOTF as the dispersive device, spectrometers can be constructed with no moving parts, having very high scanning speed, a wide spectral working range, and random wavelength access (Eilert, 1995). Compared to diffraction-grating spectrometers, the electronically tunable filter-based instruments have a much higher cost and thus are not widely used. 4.2.2 Non-dispersive systems There are three main groups of non-dispersive systems. The first group of spectrometers is based on the use of Fourier transform (FT) and the Michelson interferometer. This type of instrument is mainly used in research laboratories. The working principle of such a spectrometer enables the system to achieve excellent wavelength precision and accuracy, a very high signal-to-noise-ratio (SNR), and a relatively fast scanning speed. Since it utilizes a Michelson interferometer to create the conditions for optical interference by splitting light into two beams and then recombining them after a path difference has been introduced using a moving mirror, the system is very delicate. Therefore, its performance is sensitive to mechanical vibrations and dust. The second group of non-dispersive systems is based on a limited number of interference filters. These are the simplest and cheapest NIR instruments. Optical filters are usually chosen according to the absorption wavelengths used for the most popular applications – e.g. protein, moisture, and oil content in agricultural samples. Therefore, interference-filter based instruments are only designed for a limited range of routine analyses. The third group is the light-emitting diode (LED) based instruments. This type of instrument employs an array of LEDs as the illumination sources that emit narrow bands of NIR light. As the emitting wavelengths are predetermined, the instrument is usually dedicated to a specific series of measurements. Both LED and filter-based instruments satisfy the need for low-cost, specific applications, and portable instrumentation for field analyses. Generally, the selection of an appropriate instrumentation configuration depends on the purpose of the application. More extensive reviews on instrumentation for vibrational NIR spectroscopy can be found in, for example, Osborne et al. (1993) and Coates (1998).

4.3 Near-infrared hyperspectral imaging With the advent of electronically tunable filters and computers with immense computational power, it is now possible to acquire NIR images along with spectral data. This technique, known as hyperspectral imaging, has shown the potential to provide more information about the functional components of grain than is possible with NIRS or optical imaging alone. It can be considered an extension of multispectral imaging, where images are captured at a much smaller number of wavelengths by placing a wheel with limited number of band-pass filters in front of a camera. Multispectral imaging systems are constrained by the slow filter-switching speed and the rather large size of the filter wheel. The latest generation of wavelength filters is based on electronically


InGaAs Camera

Stand

Halogen-tungsten lamp Data storage and analysis

LCTF

Figure 15.7 Near infrared (NIR) imaging system (photograph courtesy of Canadian Wheat Board Centre of Grain Storage Research, Winnipeg, MB, Canada).

controlled liquid-crystal elements in a Lyot-type birefringent design. These liquidcrystal tunable filters (LCTF) select a transmitted wavelength range while blocking all others, providing rapid selection of any wavelength in the visible to NIR range. Such filters can be combined with charge-coupled device (CCD) cameras to create powerful spectral imaging instruments (Figure 15.7). The strengths of LCTFs include compactness, large apertures and field-of-views, low wavefront distortion, flexible throughput control, and low power requirements (Jha, 2000).

4.4 The application of NIR spectroscopy and hyperspectral imaging systems The development of a near-infrared (NIR) spectroscopy system for measuring the moisture and protein content in wheat, the kernel vitreousness or hardness, fungal contamination, scab or mould damage, and insect infestation has made the measurement of these quality factors objective, and the system has been adopted by the industry (Delwiche, 1998, 2003; Delwiche and Hruschka, 2000; Wang et al., 2002). NIR spectroscopy has replaced the chemically intensive Kjeldahl method for protein content measurement in many countries. For proper functioning of the NIR system, large amounts of reference data from different growing regions should be used for calibration. Once properly calibrated, it is a rapid technique requiring small sample sizes. NIR spectroscopy has the potential to be used for measuring the hardness and vitreousness of kernels, for color classification, the identification of damaged kernels, the detection of insect and mite infestation, and the detection of mycotoxins (Singh et al., 2006). Also, the feasibility of using reflectance characteristics for quick identification of bulk grain samples has been assessed (Mohan et al., 2004). The NIR spectroscopic method detects infested grain kernels based on differences in spectral reflectance (Dowell et al., 1999). The cuticular polysaccharides (chitin content) of insects have a different spectral reflectance from that of water, protein, starch, or other chemical constituents in the grain. This method was successfully used to identify wheat kernels infested by the larval stages of R. dominica, S. oryzae, and S. cerealella using wavelengths in the range of 1000–1350 nm and 1500–1680 nm.

Thermal imaging 369

It inspected 300 kernels in 3 minutes, and detected third and fourth instars of S. oryzae with 95 percent accuracy (Dowell et al., 1998). The unique chemical composition of the cuticle of different insect species was used to identify 11 different primary and secondary grain feeders with more than 99 percent accuracy (Dowell et al., 1999). Ridgway and Chambers (1996) determined that the NIR method detected infestation in samples containing 270 and more insects/kg of grain. By analyzing single kernels, the NIR method was able to detect infestation of S. oryzae in wheat only after the third instar stage (Dowell et al., 1998). No difference was detected between the spectra of kernels partially consumed by insects, and sound kernels (Dowell et al., 1998). Near-infrared hyperspectral imaging systems based on LCTFs have gained widespread popularity in medical imaging, but their applications in the field of agricultural products have so far been very few. Evans et al. (1998) used an LCTF-based imaging system to evaluate the vigor of bean plants at different nitrogen-stress levels. Although the authors could successfully quantify these stresses in plants, their imaging system suffered from a slow response of the LCTFs in attenuating desired wavelengths. Similar concerns were shared by Archibald et al. (1998), who developed a system to analyze wheat protein and determine color classification on a single-kernel basis. With advances in the LCTF technology and the faster computational speed of personal computers, the problem of the slow response of LCTFs has been overcome. This is evident from a recent publication by Cogdill et al. (2004), who used a similar hyperspectral imaging system and found it to have very fast wavelength tuning capability. They obtained accurate predictions for moisture concentrations but not for oil content in maize, but conceded that the errors in predicting oil content were attributable to the reference method rather than the spectrometer.

5 Thermal imaging The thermal image is generated from the infrared radiation (700–1 nm) emitted from an object at a given temperature. In other words, thermal imaging provides a surfacetemperature map of an object. The thermal imaging system (Figure 15.8) includes an infrared thermal camera (such as the ThermaCAM TM SC500, of FLIR systems, Burlington, Ontario, Canada, an un-cooled focal plane array type camera capable of generating images of 320 × 240 pixels in the spectral range 7.5–13.0 µm) and a computer system. The thermal resolution of such camera is quite high (approximately 0.07◦ C at 30◦ C). Close-up lenses (for example, of 50-µm focal length) are usually attached to the original lens of the camera (FOV 24◦ × 18◦ ) to obtain magnified thermal images of a kernel.

5.1 Application of thermal imaging Thermal imaging has been demonstrated to detect insect-infested kernels and different classes of wheat (Manickavasagan et al., 2006a, 2006b). In thermal imaging, the emitted energy is represented as a two-dimensional image. This imaging technique is a non-contact type, but it requires the creation of temperature differences in an object,


Figure 15.8 Thermal imaging system (photograph courtesy of Canadian Wheat Board Centre of Grain Storage Research, Winnipeg, MB, Canada).

either by heating or cooling, to obtain internal information. By heating or cooling kernels of wheat which were initially at a uniform temperature, it is possible to show the differences between sound and infested kernels, or kernels of different classes. At present, thermal imaging is at research stage; however, it has already shown the potential to provide information associated with wheat quality degradation.

6 Potential practical applications of machine vision technology 6.1 Automation of railcar unloading To automate the handling of the contents of a railcar, it is necessary to collect a grain sample and rapidly identify it using an imaging system. In such a situation, the grain sample can be presented in bulk and imaged. The classification accuracies from the bulk images were nearly 100 percent for five grain types. Also, the feasibility of using reflectance characteristics for quick identification of bulk grain samples was assessed. Based on these studies (Paliwal et al., 2001, 2003a, 2003b, 2005; Visen et al., 2001, 2002, 2004a, 2004b; Mohan et al., 2004), it was concluded that a system based on the analysis of bulk images could be developed for automation of railcar unloading.

6.2 Optimization of grain cleaning To automate a cleaner, it is desirable that it should have a decision support system to adjust its parameters (such as vibration rate, grain flow rate, etc.) by calculating the amount of impurity being removed from the sample. This can be done by calculating the change in the ranges of morphological features of the particles before and after

References 371

the sample is passed through the cleaner. The ranges of morphological features change significantly when a sample is passed through the cleaner, and thus can be used to provide a feedback to the system.

6.3 Quality monitoring of export grains The grain being loaded onto ships for the export market is usually blended from different bins containing grain with different degrees of cleanliness to meet the specified tolerances for foreign material by the importing customer. Using high-resolution images of kernels of five grain types (barley, CWAD wheat, CWRS wheat, oats, and rye), and five broad categories of dockage constituents (broken wheat kernels, chaff, buckwheat, wheat-heads, and canola), analyses were performed for their classification. Different feature models, viz. morphological, color, textural, and a combination of the three, were tested for their classification performances using a neural network classifier. Kernels and dockage particles with well-defined characteristics (e.g. CWRS wheat, buckwheat, and canola) showed near-perfect classification, whereas particles with irregular and undefined features (e.g. chaff and wheat-heads) were classified with accuracies of around 90 percent. The similarities in shape and size of some of the particles of chaff and wheat-heads to those of the kernels of barley and oats adversely affected the classification accuracies of the latter. With calibration, algorithms can be used to monitor and control the blending of grain.

6.4 Detection of low-level insect infestation The Berlese funnel method, currently used by the Canadian Grain Commission to detect infestations, extracted 67.2, 50.5, and 81.0 percent of first, second, and third instars of C. ferrugineus larvae, respectively, in 6 hours. The same infested kernels were all identified as being infested by the trained BPNN using the features extracted from the soft X-ray images (Karunakaran et al., 2003a, 2003b, 2003c, 2004a, 2004b, 2004c, 2005d). Potential exists to identify uninfested and infested kernels (included kernels infested by external and internal grain feeders) using soft X-rays.

Acknowledgments This chapter summarizes results from studies carried out by several research trainees, who were supervised by Dr Jayas. Their contributions are gratefully acknowledged. Sections 4.1 and 4.2 are reproduced from the MSc. thesis of Mr Wenbo Wang, who was supervised by Dr Paliwal, and we are thankful to him.

References Anonymous (1987) Official Grain Grading Guide. Winnipeg, Canadian Grain Commission. Archibald DD, Thai CN, Dowell FE (1998) Development of short-wavelength near-infrared spectral imaging system for grain color classification. Proceedings of the SPIE, 3543, 189–198.


Birth GS, Zachariah GL (1976) Spectrophotometry of agricultural products. In Quality Detection in Foods (Gaffney JJ, ed.). St Joseph: ASAE, 6–11. Campbell NA, Reece JB, Mitchell LG, Taylor MR (2002) Biology, Concepts and Connections, 4th edn. San Francisco: Benjamin Cummings. Canada Grain Act (1975) Canada grain regulations. Canada Gazette Part II, 109 (14). Coates J (1998) Vibrational spectroscopy, instrumentation for infrared and Raman spectroscopy. Applied Spectroscopy Reviews, 33 (4), 267–425. Cogdill RP, Hurburgh Jr. CR, Rippke GR (2004) Single-kernel maize analysis by nearinfrared hyperspectral imaging. Transactions of the ASAE, 47 (1), 311–320. Connolly C (2006) Machine vision developments. Sensor Review, 26 (4), 277–282. Crowe TG, Luo X, Jayas DS, Bulley NR (1997) Color line-scan imaging of cereal grain kernels. Applied Engineering in Agriculture, 13 (5), 689–694. Delwiche SR (1998) Protein content of single kernels of wheat by near-infrared reflectance spectroscopy. Journal of Cereal Science, 27, 241–254. Delwiche SR (2003) Classification of scab- and other mold-damaged wheat kernels by near-infrared reflectance spectroscopy. Transactions of the ASAE, 46 (3), 731–738. Delwiche SR, Hruschka WR (2000) Protein content of bulk wheat from near-infrared reflectance spectroscopy. Cereal Chemistry, 77 (1), 86–88. Dowell FE, Throne JE, Baker JE (1998) Automated nondestructive detection of internal insect infestation of wheat kernels using near-infrared reflectance spectroscopy. Journal of Economic Entomology, 91 (4), 899–904. Dowell FE, Throne JE, Wang D, Baker JE (1999) Identifying stored-grain insects using near-infrared spectroscopy. Journal of Economic Entomology, 92 (1), 165–169. Eilert AJ, Danley WJ, Wang X (1995) Rapid identification of organic contaminants in pretreated waste water using AOTF near-IR spectrometry. Advanced Instrumentation Control, 50 (Pt. 2), 87–95. Evans MD, Thai CN, Grant JC (1998) Development of a spectral imaging system based on a liquid crystal tunable filter. Transactions of the ASAE, 41 (6), 1845–1852. FAOSTAT (2006) FAOSTAT Classic – Crops Primary. Food and Agriculture Organization of the United Nations (available on-line at http://faostat.fao.org/site/408/default.aspx, accessed 11 November 2006). Gonzalez RC, Woods RE (1992) Digital Image Processing. Reading: Addision-Wesley. Hawk AL, Kaufmann HH, Watson CA (1970) Reflectance characteristics of various grains. Cereal Science Today, 15, 381–384. Jayas DS, Murray CE, Bulley NR (1999) An automated seed presentation device for use in machine vision identification of grain. Canadian Agricultural Engineering, 41 (2), 113–118. Jayas DS, Paliwal J, Visen NS (2000) Multi-layer neural networks for image analysis of agricultural products. Journal of Agricultural Engineering Research, 77 (2), 119–128. Jayas DS, Mohan AL, Karunakaran C (2005) Unloading automation implemented in grain industry. Resource, September, 6–7. Jha AR (2000) Infrared Technology, Applications to Electro-Optics, Photonic Devices and Sensors, 1st edn. Hoboken: Wiley-Interscience, Inc.

References 373

Karunakaran C, Visen NS, Paliwal J, Zhang G, Jayas DS, White NDG (2001) Machine vision systems for agricultural products, CSAE Paper No. 01-305, Mansonville QC, CSAE/SCGR. Karunakaran C, Jayas DS, White NDG (2003a) X-ray image analysis to detect infestations caused by insects in grain. Cereal Chemistry, 80 (5), 553–557. Karunakaran C, Jayas DS, White NDG (2003b) Soft X-ray image analysis to detect wheat kernels damaged by Plodia interpunctella (Lepidoptera, Pyralidae). Sciences des Aliments, 23 (5–6), 623–631. Karunakaran C, Jayas DS, White NDG (2003c) Soft X-ray inspection of wheat kernels infested by Sitophilus oryzae. Transactions of the ASAE, 46 (3), 739–745. Karunakaran C, Jayas DS, White NDG (2004a) Soft X-rays, a potential insect detection method in the grain handling facilities. In International Quality Grains Conference, Indianapolis, IN, July 19–22. Karunakaran C, Jayas DS, White NDG (2004b) Detection of infestations by Cryptolestes ferrugineus inside wheat kernels using a soft X-ray method. Canadian Biosystems Engineering, 46, 7.1–7.9. Karunakaran C, Jayas DS, White NDG (2004c) Detection of internal wheat seed infestation by Rhyzopertha dominica using X-ray imaging. Journal of Stored Products Research, 40, 507–516. Karunakaran C, Jayas DS, White NDG (2004d) Identification of wheat kernels damaged by the red flour beetle using X-ray images. Biosystems Engineering, 87 (3), 267–274. Keefe PD (1992) A dedicated wheat grain image analyser. Plant Varieties and Seeds, 5 (1), 27–33. Keefe PD, Draper SR (1986) The measurements of new characters for cultivar identification in wheat using machine vision. Seed Science and Technology, 14, 715–724. Kubelka P, Munk F (1931) Ein Beitrag zur Optik der Farbanstriche. Zeitshrift für Technische Physik, 12, 593–604. Luo X, Jayas DS, Crowe TG, Bulley NR (1997) Evaluation of light sources for machine vision. Canadian Agricultural Engineering, 39 (4), 309–315. Luo XY, Jayas DS, Symons SJ (1999a) Comparison of statistical and neural network methods for classification of cereal grains using machine vision. Transactions of the ASAE, 42 (2), 413–419. Luo XY, Jayas DS, Symons SJ (1999b) Identification of damaged kernels in wheat using a color machine vision system. Journal of Cereal Science, 30 (1), 49–59. Majumdar S, Jayas DS (1999a) Single-kernel mass determination for grain inspection using machine vision. Applied Engineering in Agriculture, 15 (4), 357–362. Majumdar S, Jayas DS (1999b) Classification of bulk samples of cereal grains using machine vision. Journal of Agricultural Engineering Research, 73, 35–47. Majumdar S, Jayas DS (2000a) Classification of cereal grains using machine vision, I. Morphology models. Transactions of the ASAE, 43 (6), 1669–1675. Majumdar S, Jayas DS (2000b) Classification of cereal grains using machine vision, II. Color models. Transactions of the ASAE, 43 (6), 1677–1680. Majumdar S, Jayas DS (2000c) Classification of cereal grains using machine vision, III. Texture models. Transactions of the ASAE, 43 (6), 1681–1687.


Majumdar S, Jayas DS (2000d) Classification of cereal grains using machine vision, IV. Morphology, color, and texture models. Transactions of the ASAE, 43 (6), 1689–1694. Majumdar S, Jayas DS, Symons SJ (1999) Textural features for grain identification. Agricultural Engineering Journal, 8 (4), 213–222. Majumdar S, Jayas DS, Hehn JL, Bulley NR (1996a) Classification of various grains using optical properties. Canadian Agricultural Engineering, 38 (2), 139–144. Majumdar S, Luo X, Jayas DS (1996b) Image processing and its applications in food process control. In Computerized Control Systems in the Food Industry (Mittal GS, ed.). New York: Marcel Dekker, Inc., pp. 207–234. Manickavasagan A, Jayas DS, White NDG (2006a) Thermal imaging to detect infestation by Cryptolestes ferrugineus inside wheat kernels. Journal of Stored Products Research (submitted). Manickavasagan A, Jayas DS, White NDG, Paliwal J (2006b) Wheat class identification using thermal imaging. Transactions of the ASABE (submitted). Melvin S, Karunakaran C, Jayas DS, White NDG (2003) Design and development of a grain kernel singulation device. Canadian Biosystems Engineering, 45, 3.1–3.3. Mohan AL, Jayas DS, White NDG, Karunakaran C (2004) Classification of bulk oil seeds, speciality seeds and pulses using their reflectance characteristics. In Proceedings of International Quality Grains Conference, Indianapolis, IN. Murray I, Williams PC (1990) Chemical principles of near-infrared technology. In Nearinfrared Technology in the Agricultural and Food Industries (Williams PC, Norris KH, eds). St Paul: American Association of Cereal Chemists, Inc., pp. 17–34. Myers DG, Edsall KJ (1989) The application of image processing techniques to the identification of Australian wheat varieties. Plant Varieties and Seeds, 2, 109–116. Nair M, Jayas DS (1998) Dockage identification in wheat using machine vision. Canadian Agricultural Engineering, 40 (4), 293–298. Neuman MR, Sapirstein HD, Shwedyk E, Bushuk W (1987) Discrimination of wheat class variety by digital image analysis of whole grain samples. Journal of Cereal Science, 6 (2), 125–132. Neuman MR, Sapirstein HD, Shwedyk E, Bushuk W (1989a) Wheat grain color analysis by digital image processing. I. Methodology. Journal of Cereal Science, 10, 175–182. Neuman MR, Sapirstein HD, Shwedyk E, Bushuk W (1989b) Wheat grain color analysis by digital image processing. II. Wheat class discrimination. Journal of Cereal Science, 10, 183–188. Osborne BG, Fearn T, Hindle PH (1993) Practical NIR Spectroscopy, with Applications in Food and Beverage Analysis, 2nd edn. New York: John Wiley & Sons. Paliwal J, Shashidhar NS, Jayas DS (1999) Grain kernel identification using kernel signature. Transactions of the ASAE, 42 (6), 1921–1924. Paliwal J, Visen NS, Jayas DS (2001) Evaluation of neural network architectures for cereal grain classification using morphological features. Journal of Agricultural Engineering Research, 79 (4), 361–370. Paliwal J, Visen NS, Jayas DS, White NDG (2003a) Cereal grain and dockage identification using machine vision. Biosystems Engineering, 85 (1), 51–57.

References 375

Paliwal J, Visen NS, Jayas DS, White NDG (2003b) Comparison of a neural network and a non-parametric classifier for grain kernel identification. Biosystems Engineering, 85 (4), 405–413. Paliwal J, Borhan MS, Jayas DS (2004a) Classification of cereal grains using a flatbed scanner. Canadian Biosystems Engineering, 46, 3.1–3.5. Paliwal J, Jayas DS, Visen NS, White NDG (2004b) Feasibility of a machine-vision based grain cleaner. Applied Engineering in Agriculture, 20 (2), 245–248. Paliwal J, Visen NS, Jayas DS, White NDG (2005) Quantification of variations in machinevision-computed features of cereal grains. Canadian Biosystems Engineering, 47, 7.1–7.6. Ridgway C, Chambers J (1996) Detection of external and internal insect infestation in wheat by near-infrared reflectance spectroscopy. Journal of Science in Food and Agriculture, 71 (2), 251–264. Sapirstein HD, Bushuk W (1989) Quantitative determination of foreign material and vitreosity in wheat by digital image analysis. In ICC ’89 Symposium, Wheat End-Use Properties (Salovaara H, ed.), Lahiti, Finland, pp. 453–474. Shashidhar NS, Jayas DS, Crowe TG, Bulley NR (1997) Processing of digital images of touching kernels by ellipse fitting. Canadian Agricultural Engineering, 39 (2), 139–142. Shatadal P, Jayas DS, Hehn JL, Bulley NR (1995a) Seed classification using machine vision. Canadian Agricultural Engineering, 37 (3), 163–167. Shatadal P, Jayas DS, Bulley NR (1995b) Digital image analysis for software separation and classification of touching grains, I. Disconnect algorithm. Transactions of the ASAE, 38 (2), 635–643. Shatadal P, Jayas DS, Bulley NR (1995c) Digital image analysis for software separation and classification of touching grains, II. Classification. Transactions of the ASAE, 38 (2), 645–649. Singh CB, Paliwal J, Jayas DS, White NDG (2006) Near-infrared spectroscopy, applications in the grain industry. CSBE Paper No. 06-189, Canadian Society for Bioengineers, Winnipeg, MB. Swinehart DJ (1972) The Beer-Lambert law. Journal of Chemical Education, 39 (7), 333–335. Symons SJ, Fulcher RG (1988a) Relationship between oat kernel weight and milling yield. Journal of Cereal Science, 7, 215–217. Symons SJ, Fulcher RG (1988b) Determination of variation in oat kernel morphology by digital image analysis. Journal of Cereal Science, 7, 219–228. Symons SJ, Fulcher RG (1988c) Determination of wheat kernel morphological variation by digital image analysis, I. Variation in Eastern Canadian milling quality wheats. Journal of Cereal Science, 8, 210–218. Symons SJ, Fulcher RG (1988d) Determination of wheat kernel morphological variation by digital image analysis, II. Variation in cultivars of white winter wheats. Journal of Cereal Science, 8, 219–229. Visen NS (2002) Machine vision based grain handling system. PhD Thesis, Department of Biosystems Engineering, University of Manitoba, Winnipeg, Manitoba, Canada.


Visen NS, Shashidhar NS, Paliwal J, Jayas DS (2001) Identification and segmentation of occluding groups of grain kernels in a grain sample image. Journal of Agricultural Engineering Research, 79 (2), 159–166. Visen NS, Jayas DS, Paliwal J, White NDG (2002) Specialist neural networks for cereal grain classification. Biosystems Engineering, 82 (2), 151–159. Visen NS, Jayas DS, Paliwal J, White NDG (2004a) Comparison of two neural network architectures for classification of singulated cereal grains. Canadian Biosystems Engineering, 46, 3.7–3.14. Visen NS, Paliwal J, Jayas DS, White NDG (2004b) Image analysis of bulk grain samples using neural networks. Canadian Biosystems Engineering, 46, 7.11–7.15. Wang D, Dowell FE, Dempster R (2002) Determining vitreous subclass of hard red spring wheat using visible/near-infrared spectroscopy. Cereal Chemistry, 79 (3), 418–422. Wang W (2005) Design and evaluation of a visible-to-near-infrared spectrograph for grain quality assessment. MSc Thesis, Department of Biosystems Engineering, University of Manitoba, Winnipeg, Manitoba, Canada. Wang W, Paliwal J (2006) Separation and identification of touching kernels and dockage components in digital images. Canadian Biosystems Engineering, 48, 7.1–7.7. Zayas I, Lai FS, Pomeranz Y (1986) Discrimination between wheat classes and varieties by image analysis. Cereal Chemistry, 63, 52–56. Zayas I, Pomeranz Y, Lai FS (1989) Discrimination of wheat and non-wheat components in grain samples by image analysis. Cereal Chemistry, 66, 233–237. Zhang G, Jayas DS, White NDG (2005) Separation of touching grain kernels in an image by ellipse fitting algorithm. Biosystems Engineering, 92 (2), 135–142.

Quality Evaluation of Rice Yukiharu Ogawa Faculty of Horticulture, Chiba University, Matsudo, Chiba, 271-8510 Japan

1 Introduction Rice (Oryza sativa L.) is one of the major commercial cereal grains worldwide, along with wheat and corn. In the order of 628 million tonnes of rice were produced throughout the world in 2005, and the world trade in the commodity that same year was 29.9 million tonnes, as estimated by the FAO (2006). Over 90 percent of rice is produced and consumed in Asia. Since the mapping of the rice genome began, genetic studies, such as genome research into rice, have progressed. Rice is therefore currently studied in many academic fields, including plant, breeding, crop, and food science. Although the aims of rice studies vary, quality evaluation of the grain as a foodstuff is one of the main goals. Computer vision technology, which is progressing all the time with the continuous development in both hardware and software, can contribute to such quality evaluation by assessing the quality of the rice grains objectively, consistently, and quantitatively. In this chapter, various techniques and methods for the quality evaluation of rice using computer vision technology are described. Rice research has various aspects, as mentioned above, and the significance of the rice quality differs within each – for example, the quality of rice as a foodstuff is different from that as a raw material. An outline of rice quality is thus described in the next section. Rice as a raw material (“raw rice”) and as a prepared foodstuff (“cooked rice”) is classified in the following sections and described together with the different evaluation techniques.

2 Quality of rice The word “quality” is extremely abstract. Consequently, before describing its evaluation, the quality of rice has to be defined. Parameters, which must be expressed by actual and measurable objects or properties, are also required to evaluate the quality. Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


16

378 Quality Evaluation of Rice

Rice grows in a paddy field and is harvested as a plant seed material. The harvested rice seed undergoes post-harvest processing, including drying, storing, and hulling. The hulled rice is commonly known as brown rice. Usually, the brown rice grain is milled to remove its bran layer, including the pericarp, testa, aleurone layer, etc. This process is also called polishing. The milled rice appears as a white, semi-transparent grain. Unlike other cereal grains, rice is usually purchased primarily as the milled product by the consumer and is consumed as a steamed or boiled product – cooked rice. Thus, the rice quality is related to which stage the rice is at. Rice as a plant seed material, brown grain, and milled product can be regarded as the raw material of foodstuffs. The parameters for the quality evaluation of rice as a raw material are therefore concerned with biological and post-harvest handling properties. These properties are influenced by characteristics under genetic control, environmental conditions, and processing conditions. Basically, it can be considered that physical properties (i.e. measurable dimensions such as the grain size, shape, and color variance, etc.), which are concerned with cultivar and growth environment, are primary parameters for the quality evaluation of raw rice (Hoshikawa, 1993a). Defects and fissures, which are associated with post-harvest processing and market value, are also regarded as parameters. Moreover, the water content and distribution within the raw grain also influence its storage properties. The chemical contents and distribution in the grain related to the morphological, histological, and structural properties are also considered in assessing raw rice quality. The content of chemical compounds also defines the nutritional quality. The aroma is related to the compounds, and is thus another quality attribute of rice, although it cannot currently be represented as a visual parameter. Steamed or boiled rice as a cooked product is a foodstuff, and therefore its quality is based on its eating quality, which is related to the physical, chemical, and physicochemical properties of the cooked grain. Among these properties, the texture of cooked rice products is one of the most important properties and it has been usually measured using sensory analysis. Cooked rice products have a high moisture content, and their starch granules are gelatinized by boiled water during cooking. In general, starch gelatinization is related to the physicochemical properties, which influence the cooked rice texture. Consequently, the water distribution in a cooked grain during cooking is an important parameter for the quality evaluation of cooked rice. The starch granules in the individual grains gelatinize and the grain shape swells with such gelatinization. As a result, the grain-scale macro-structure changes drastically during cooking, and such structural changes to the interior and exterior of the grain are reflected in the rice texture. Accordingly, the structural properties of the cooked grain, including its surface structure (which is concerned with appearance), are also important parameters for quality evaluation. The rice-grain interior consists mostly of starch and starch granules. The starch in a grain is enveloped in endosperm cells, which are composed of cell-wall materials. Histological microstructures, such as cell formation and distribution, must therefore be related to the physical and physicochemical properties, such as hardness and stickiness. The cell-scale micro-structure of the cooked grain is an important parameter for quality evaluation. The thermal condition, aroma, taste, etc. can also be regarded as quality evaluation parameters for the cooked rice grain, although these have not so far been visualized.

Quality evaluation of raw rice 379

3 Quality evaluation of raw rice 3.1 Physical properties Visual inspection of the grain by human eyes is a primary method of grain quality inspection commercially. Automated inspection equipment and methods are important and are in demand. Computer-aided machine vision systems can provide objective, consistent, and quantitative measurements. They can also automatically and accurately inspect visual qualities such as grain contour, size, color variance and distribution, and damage. Image-processing techniques for computer-aid machine vision systems have been developed, for example for determining the physical dimensions of milled kernel (Goodman and Rao, 1984). Pattern-recognition techniques can also be used as an aid in grain characterization, and can be an effective method for identifying and classifying the grains (Lai et al., 1986). Sakai et al. (1996) demonstrated the use of two-dimensional image analysis for the determination of the shape of brown and polished rice grains of four varieties. The sample grains were polished by three different polishing methods. The rice varieties were well separated by image analysis using suitable dimension and shape factors, whereas the grain polished by different methods could not be differentiated accurately. The water condensation on the grain surface is caused by changes in the environmental conditions of temperature and relative humidity during storage, and leads to a deterioration in quality. Atungulu et al. (2003) investigated the relationship between the amount of condensed water, estimated by thermodynamic simulation and experimental results, and the value obtained from color indices such as RGB and/or HSI in the resulting grain images. They concluded that the deviations from initial hue and intensity of the HSI indices were changed by the condensation on grain surface, and were also related to the surrounding environment of grains, such as temperature and relative humidity. In general, an individual object should be placed under a camera for image processing, and clear images must be provided for the machine vision system. Boundary extraction and geometrical feature measurements on physically touching objects are therefore classic problems when a real-time machine vision inspection system is performed. The touching objects yield connected regions in the image after segmentation from the background, thus making measurements of individual objects impossible without further pre-processing. Considering this, Shatadal et al. (1995a) developed an algorithm to segment connected grain-kernel image regions. Their algorithm used the image transformed by the discipline of mathematical morphology, and succeeded in separating rice kernels in the image. The geometrical features were also extracted from both software-separated and physically-separated kernels for pattern classification (Shatadal et al., 1995b). The authors described that there was an important limitation of the algorithm, which led to failure when the connected kernels formed a relatively long isthmus or bridge. Later, Wang and Chou (2004) developed a more efficient method to segment touching rice kernels using the active contour model and the inverse gradient vector flow field. The inverse gradient vector flow field was first proposed to automatically generate a field center for every individual rice kernel in an image. These centers were then employed as the reference for setting initial deformable


contours that were required for building an active contour model. It was found that the complete contours of touching objects identified by this approach could facilitate subsequent image processing to obtain the geometric, texture, and color characteristics of objects. An automatic inspection system with multiple kernel image inspection can speed up the process. However, as mentioned above, to present many tiny grain kernels in an oriented form for machine vision inspection is not an easy task, and to calculate grading parameters of every kernel with many kernels touching each other randomly is very difficult and time-consuming. Therefore, an efficient device is required to present multiple grain kernels that are not in contact with one another, when an automatic grain quality inspection system with high performance is developed. Wan (2002) developed an automatic kernel-handling system, consisting of an automatic inspection machine and an image-processing unit. His system could continually present matrix-positioned grain kernels to charge-coupled device (CCD) cameras, singularize each kernel image from the background, and discharge kernels to assigned containers. The inspection machine had scattering and positioning devices, a photographing station, a parallel discharging device, and a continuous conveyer belt with carrying holes for the grain kernels. The image-processing unit and the inspection machine were designed to work concurrently to provide high throughput of individual kernel images. Wan et al. (2002) also investigated aspects associated with the performance of his automatic quality inspection system for evaluating various rice appearance characteristics, such as sound, cracked, chalky, immature, dead, broken, and damaged. Carter et al. (2006) proposed both cluster and discriminant analyses for establishing the suitability of the measured parameters for authentication of granular food using his developed digital imaging system and fuzzy logic. Results demonstrated that it might be possible to distinguish between different varieties of the same rice. Milling of rough rice is usually conducted to produce white and polished edible grain, owing to consumer preference. The important parameters to evaluate milled rice quality are grain size and shape, whiteness, and cleanliness, which are correlated with the transaction price of the rice. These factors are closely related to the process of milling, in which rough rice is first subjected to dehusking or removal of hulls, and then to the removal of the brownish outer bran layer. Finally, polishing is carried out to remove the bran particles and to provide surface gloss to the edible white portion. The degree of milling determines the extent of removal of the bran layer from the surface of the milled kernels, and is thus related to the whiteness of rice. Yadav and Jindal (2001) developed techniques that could be used for estimating the head rice yield, in which the weight percentage of the milled kernels was represented by three-fourths of their original length of brown rice relative to the rough rice weight and the degree of milling, based on two-dimensional imaging of milled rice kernels. There quantity of broken rice kernels allowed is specified when buying milled rice, and broken rice kernels have normally only half the value of whole or head rice. The weight percentage of whole kernels remaining after milling is one of the important physical characteristics that determine the rice quality. The amount of broken rice kernels is determined mainly by visual selection of these kernels from a large quantity of rice. The length and width of rice kernels is generally measured using a single caliper. These analyses can be


performed much faster and more accurately using machine vision systems. Dalen (2004) reported that the size and size distribution of rice and the amount of broken rice kernels could be determined by image analysis using flatbed scanning. He demonstrated that his flatbed scanner with image analysis was a fast, easy, and low-cost method of determination. Rapid moisture adsorption by low-moisture rice grains may cause the grains to fissure, which becomes a cause of deterioration for the eating quality of rice when cooked. The combination of physical properties such as stress and strain has a continuous influence on the rice grain. Such stress-fissured kernels break more readily than sound kernels during harvesting, handling, and milling, and thereby reduce the quality and the market value of the grain. Stress-fissure detection is still an important task in rice quality evaluation. Lan et al. (2002) developed a machine vision system with a CCD black/white camera, an image frame-grabber, a computer, and an image-processing program to obtain images of fissured rice grains. The fissure pattern difference between long and medium grain rice was recognized after analyzing the processed images of fissured grains. The detection procedure for fissures in a sample kernel of the medium rice was carried out by image-processing methods, such as gamma correction, histogram equalization, erosion, regional enhancement, and edge detection. For the long kernels, gamma correction, high-pass filtering, contrast adjustment, and regional enhancement were carried out because differences in the fissure pattern between long and medium rice kernels were observed. Their computer vision system was able to reveal 94 percent of all the fissure lines detected in medium grains by a human expert, and 100 percent in long grains. The stress distribution inside a rice kernel, which is the origin of kernel fissures, is difficult to estimate by the imaging method. A finite-element analysis method can simulate such stress distributions in a kernel by computer. Jia et al. (2002) mapped and analyzed the distributions of radial, axial, tangential, and shear stresses in a kernel during drying by the finite-element simulation combined with high-speed microscopy imaging of the fissure appearance. As a result, they found that two distinct stress zones existed inside a rice kernel during drying – a tensile zone near the surface, and a compressive zone close to the center. It was also found that, as drying proceeded, radial, tangential, and shear stresses gradually approached zero in magnitude and became neutral in direction after 60 minutes of drying at 60◦ C and 17 percent relative humidity. Only axial stress remained at a pronounced level, even after drying, which helped to explain the fact that most fissures propagate perpendicular to the longitudinal axis of the rice kernel.

3.2 Water content and distribution Water distribution in a rice kernel is one of the important parameters for the quality evaluation of rice grains. The changes in the water distribution of a rice seed during morphological development as a plant material are also important to elucidate the quality of matured grain after harvest, because starch and other compounds are accumulated by assimilating transport with water during development. In order to visualize such morphological development of rice caryopsis by adding information on the moisture


distribution pattern, nuclear magnetic resonance (NMR) micro-imaging was applied as a non-destructive measurement technique (Horigane et al., 2001). In general, the NMR imaging technique can be used for the non-destructive and non-invasive determination of moisture distribution and mobility in a grain. Thus, the moisture distribution pattern can be discussed in relation to the morphological development. The moisture distribution images of young tissue, the pericarp vascular bundle, and the endosperm up to 25 days after anthesis were obtained, and the route for water supply to or drain from the embryo was observed in their study. The three-dimensional structure of developing spikelets was represented as a maximum-intensity projection image, which is a reconstructed image of a transparent view through an object using three-dimensional data. By the observation of such images, the moisture content of the older caryopses was found to be smaller than the younger during development because the resulting signal intensities in the maximum intensity projection images were decreased. The increments in width and thickness of the caryopsis and the junction between palea and lemma were observed by cross-sectional NMR images. These findings supported the fact that water flows from the pericarp vascular bundle into the nucellus. The water content of rice seed that has been harvested as an agricultural product is removed by drying. The drying process thus greatly affects the rice quality. Ishida et al. (2004) visualized changes in the water distribution in a rough rice seed during drying by the single point mapping imaging technique combined with magnetic resonance imaging (MRI). They traced the decrease of water in rice seeds after harvesting at various drying temperatures, and compared the decrease in image intensity, which was proportional to the removable water content, with the grains dried by the ovendrying method. In experimental results, they showed that the water content in fully ripened seeds was approximately 20 percent. It was a low moisture content compared with usual seeds because physiological drying of the husk occurs before harvesting. With a water content of less than 20 percent, MRI images could not be obtained by the spin-echo method; however, it was possible to obtain images in a short time by the single point mapping imaging method, and thus the process of water reduction was traceable. Water in this concentration range was adsorbed water on the surface of molecular structures in the grain, while water content of less than 7 percent was tightly bound. In their results, the water presented mainly in the grain kernels but not in the husk, and embryos contained rather large amounts of water. The signal of the water in the images was reduced as the drying time elapsed and the drying temperature was increased. It disappeared uniformly from all areas of the endosperm of the seeds. The moisture content and its distribution in a rice grain were also observed by a method based on electromagnetic imaging (Lim et al., 2003). Because the dielectric constant of water is much greater than that of the dry material of grain, the dielectric constant of grain is correlated with its water content. Such correlation forms the basis for rapid determination of the moisture content of rice grain electrically. Even though this provides relatively reliable moisture measurement, it generally lacks spatial resolution. The method based on electromagnetic imaging mapped the two-dimensional moisture distribution in rice grains, and a quantitative image reconstruction algorithm with simultaneous iterative reconstruction was employed to achieve rapid convergence of a final acceptance solution.


3.3 Compound contents and distribution 3.3.1 Microscopic imaging The chemical-compound content and distribution in a grain, which affects the morphological and histological characteristics, is generally fixed in the growth stage and has an influence on the rice quality. Illustrations and/or photographs were used to represent such physical structures of rice until computer vision was developed (Bechtel and Pomeranz, 1978a). The internal structures of the grain were also determined precisely by hand (Hoshikawa, 1993b). Histological microscopy of the grain in which those results were printed as photographs described the anatomical nature of rice seed as a plant material. Electron micrographs, including scanning electron micrographs (SEM), are usually used to observe the ultrastructure in micrometer-scale samples. For example, Bechtel and Pomeranz (1977, 1978b, 1978c) captured the ultrastructure of the mature ungerminated rice caryopsis by light- and electron-microscopy. Watson and Dikeman (1977) also employed SEM for observations of the endosperm, aleurone, germ, and hull of the grain, with the objective of obtaining a better understanding of rice ultrastructure. Such anatomical studies contribute to the inspection of biochemical properties and structural characteristics, which influence the availability of rice’s nutrients as a foodstuff. In other words, the nutritional quality of foodstuffs is directly related to the nature of nutrient storage in the grains, which can be observed using a microscope (Yiu, 1993). A microscope can visualize the structural details that are required for analyzing histological characteristics, and can also obtain image data by the use of digital capturing apparatus. The anatomical and histological structures, which can be analyzed by computer vision, are also related to the physical properties. For the measurement of such microstructures, with the internal chemical-compound content and distribution of a biological material, a traditional sectioning technique using a microtome has been applied. Although this technique is destructive, the distribution of various chemical compounds and their roughly quantitative values in a section can be visualized and analyzed in two dimensions by suitable staining and/or various imaging methods. There are lots of studies for observing the histological components in a small segment of rice grain by light microscopy with histochemical staining. However, whole-size sections of a rice grain, which must be of high quality for the observation of microscopy, have not been obtained by a standard sectioning method, because the rice grain has poor mechanical properties for sectioning and low infiltration properties for an embedding matrix such as paraffin. Moreover, the moisture content of rice grains is too low for the collection of frozen sections. Furukawa et al. (2003) demonstrated that the cross-sectional images of rice kernels were stained and observed by light microscopy and confocal laser scanning microscopy. Rice cross-sections of 200 µm were obtained using a laser blade and microtome. They then applied immunofluorescence labeling with specific antibodies as a histological staining technique, for visualization of the distribution of proteins stored in endosperm tissues. As a result, localization of two types of protein bodies in endosperm tissue was observed. It was also found that low-glutelin rice was different from the other cultivars not only in the major storage protein composition, but also in the distribution of storage proteins in endosperm tissue.


A special sectioning method using cellulose tape was proposed by Palmgren (1954) for the study of large, hard, and brittle specimens. The adhesive-tape method facilitated preparation and improved the quality of the resulting sections of the whole body of a baby rat, which could then be stained for histological and histochemical characteristics (Kawamoto and Shimizu, 1986). Ogawa et al. (2003a) employed the adhesive-tape method, combined with a better preparation technique for preserving microstructural details, to obtain whole rice-kernel sections. This method was a combination of tape-aided sectioning on a standard microtome and an autofluorescence visualization technique by microscope in the ultraviolet (UV) range to observe the histological properties of the whole size and the complete shape of the rice section. The procedure of tape-aid sectioning for this method is as follows: 1. The sample is dehydrated in a graded ethanol series followed by xylene, transferred to melted paraffin, allowed to infiltrate, and embedded by hardening of the paraffin. 2. Embedded rice kernels are sectioned by the usual microtome, at ambient temperature, equipped with disposable blades. Each kernel is sectioned until the desired portion is exposed. A piece of adhesive tape is then firmly pressed to the face of a specimen block. While holding the tape, the microtome is advanced to cut a section stuck to the tape. Note that the adhesive tape is a special product made of polyester coated with a solvent-type acrylic resin that serves as an adhesive. 3. The tape-section is affixed to a glass slide with the specimen side facing up, and is deparaffinized in xylene. Afterwards, microscopic observable sections can easily be obtained. 3.3.2 Virtual visualization To observe the internal composition in three dimensions, Levinthal and Ware (1972) developed a three-dimensional reconstruction technique using serial section images and interactive formation. This technique was applied to measure the three-dimensional physical structure of the central nervous system of a simple animal. The interactive formation method for three-dimensional reconstruction is based on the outline of the sectioned objects constructed, and thus could not be reconstructed precisely. Ogawa et al. (2000, 2001) developed a modified three-dimensional reconstruction and visualization technique. This technique is a combination of tape-aided serial sectioning, staining and digital imaging, and virtual rendering by computerized reconstruction. The concept of this technique is embodied in the schematic diagram presented in Figure 16.1. By tape-aided microtome sectioning, it was found that a set of serial sections of a rice grain could be prepared and preserved with their own set of relative position data. Two positioning rods were also carefully embedded with their long axes perpendicular to the bottom plane of the embedding mold and sectioned with the sample, as shown in Figure 16.1. After sectioning, a single set of serial sections was stained by a suitable histochemical dye and was captured in by a charge-coupled device (CCD) camera. As the stained areas represented areas containing a dye–target complex, the distribution of each compound in the section was visualized in two dimensions. Since all sections of the sample grain were stuck to adhesive tape and the positioning rods,


Knife blade of microtome Embedded material including positioning rods

Stained section image

2D image

CCD camera

Reconstructed 3D model (stacked 2D images)

Stereomicroscope

Adhesive tape

-Staining -

Position adjustment markers Stained sections

Obtained sections Reconstructed positioning rods 1. Serial sectioning

2. Digital imaging

3. Virtual rendering

Figure 16.1 Schematic diagram of the virtual three-dimensional reconstruction and visualization technique using the tape-aid serial sections with position adjustment markers.

the relative position of each serial section could be adjusted by referencing the position adjustment markers if the captured position of the section images differed from one to another. All adjusted section images of a set of serial sections were stacked in the memory of a personal computer to produce a three-dimensional plotting model, using the volume-rendering method. The distribution of various compounds in a rice kernel could be visualized in a virtual three-dimensional model. Figure 16.2 shows images of a sample section and its stained result. The thickness of the section was 10 µm. A rice section and sliced positioning rods as position adjustment markers were stuck to adhesive tape, and therefore the relative position between the serial sections could be adjusted as described above. A double-staining method with a combination stain of coomassie brilliant blue (CBB) solution and iodine solution was applied to the section for the visualization of protein and starch distribution (Figure 16.2b); protein was stained blue by CBB, and starch was stained purple or brown by iodine solution. The compound distributions on the section, which was clearly differentiated by color, were visualized. Protein was mainly distributed around the edges of the section, while starch was distributed in the inner area. Figure 16.3 shows a three-dimensional plotting image reconstructed from a set of double-stained serial sections in a personal computer. The distributions of the stained protein and starch compounds in a rice grain were visualized in three dimensions. The embryo of the grain pointed towards the top, and a 1/10 opacity ratio was employed. Since the three-dimensional plotting model was reconstructed using the volume-rendering method, the voxel data, which were produced by pixel data and the thickness of the sections, could represent the position of compounds in the plotting model. The size of one voxel of this model was 10 × 13 × 13 µm, because the pixel size of the two-dimensional images was 13 × 13 µm and the section thickness was 10 µm. Therefore, each voxel represented not only the shape of the grain as drawn by a polygon, but also positional information regarding compounds as voxel data, and this could be virtually subtracted by data-processing techniques.


c

a

b

Figure 16.2 Images of (a) a sample section and (b) its stained result. The circular areas (c) are the position adjustment markers, and their thickness was 10 µm. These sections were stuck to an adhesive tape, and thus the relative position of each serial section could be adjusted by referencing the position adjustment markers if the captured position varied. Staining was performed by soaking in 0.05-N iodine solution for 30 seconds and washing with distilled water, then soaking in CBB solution for 30 seconds and washing with removal solution. The magnification bar is 1 mm.

a

Figure 16.3 An image of a three-dimensional plotting model reconstructed from a serial of double-stained sections in a personal computer. The two positioning rods (a) are also reconstructed from position adjustment markers. The volume-rendering method is used for this reconstruction technique. The wrinkles of the plotting images, which look like the contour lines of a contour map, are caused by section thickness. This is a peculiarity of this visualization technique.

Figure 16.4 shows virtually divided images of the distribution of protein and starch in a sample rice grain. These images are extracted from that in Figure 16.3, based on the color differences in compound staining. Consequently, it can be visualized and observed that the protein, represented by the dark areas in Figure 16.4, is located in the surrounding parts of the grain and embryo. Starch is located in the interior portions. Because this three-dimensional visualization technique is based on histochemical technology, it can visualize the distribution of various compounds in a rice grain.


(a)

(b)

Figure 16.4 Virtually divided images of the distributions of protein and starch in a sample rice grain: (a) protein is distributed at the outer parts of the grain and embryo; (b) starch is located in the interior portions.

Ogawa et al. (2002a) also developed another three-dimensional visualizing technique for the observation of rice-seed structure in three dimensions. A three-dimensional internal structure observation system (3D-ISOS, Toshiba Machine Co. Ltd, Numazu, Japan) was applied to observe a rice caryopses structure during developing. This system can slice a sample material sequentially and capture each cross-section using a color CCD imaging device (DXC-930, Sony Co., Tokyo, Japan), incorporating uniform lighting conditions. Because the captured images of the cross-section are sequentially digitized, they can be virtually stacked in a personal computer using the volumerendering method. As a result, the three-dimensional structure of the sample material can be visualized by displaying the stacked image set. To obtain samples of dyed rice seed, a cut stem bearing a panicle, collected 30 days after flowering (before the fullyripe stage), was placed in a 0.1% rhodamine B solution in distilled water for 2 days to imbibe the dye. Figure 16.5 shows images of the resulting three-dimensional model of the rice seed produced by the virtually stacking of the serial image set of the crosssections. Figure 16.5a shows the image of the simple three-dimensional model, while Figure 16.5b represents a three-dimensional form of the vascular bundle. This model was extracted from Figure 16.5a by image processing to suppress the green and white voxels. Using this technique, the three-dimensional structure of the vascular bundles can be observed by color extraction based on natural pigmentation or artificial dyeing. Ogawa et al. (2002b) also determined the lipid distribution of a brown rice kernel in three dimensions by application of the tape-aided sectioning technique. Lipid is one of the major constituents of rice grain, and its distribution is not uniform in the brown rice kernel, as measured by chemical analysis for the graded milling flours (Kennedy et al., 1974). It was also reported that the outer layer of rice kernels, which was in the bran including the germ, had larger amounts of lipid than did the inner parts, i.e. the core or inner endosperm. Stored rice, especially that stored for an extended period after harvesting, does not have a pleasant odor when it is cooked. This odor is linked


c d e

g

f

a

h

b

Figure 16.5 Images of (a) the simple three-dimensional model and (b) the three-dimensional form of the vascular bundle of the rice seed structure. The vascular bundle (c), dyed red by rhodamine B, appears as a cage-like structure (note that this is red in the actual model). Because the seed was immature, the hull, including the palea (d) and the lemma (e), is represented as paler parts (green in the actual image). The rachilla (f) and upper (g) and lower (h) glume are also shown. The rhodamine B solution is distributed through the vascular bundles by diffusion. Usually the rice seed has eight vascular bundles, but only six are shown in the resulting image because two pairs of vascular bundles at the interlocking edges of the palea and lemma are merged by the diffusion of the dye.

to the enzyme reactions and/or lipid autoxidation (Yasumatsu and Moritaka, 1964). Lipid autoxidation, which influences the rice quality, is immediately triggered by air contact. The lipids that are located in the outer area of a kernel are considered to be more oxidizable than those in the inner parts. The observation of lipid distribution in three dimensions is thus a significant improvement to the research carried out to this point. In general, histochemical techniques have been applied for the observation of chemical distributions in a section. In order to obtain sections from a material, paraffin is commonly used as the embedding material. Consequently, paraffinization and deparaffinization steps are required, including a xylene-soaking process. As not only paraffin but also the lipid content is removed from the thin section by the xylene-soaking process, the common paraffin-embedding method is not suitable for the observation of the real lipid distribution in a grain section. Although the resin-embedding methods using polymeric resin, or frozen-section methods for materials with high water content have been used for lipid observation, a small piece of chopped specimen is needed to obtain sections. Therefore, the lipid distribution in an area as large as a whole rice section is difficult to measure by the usual histochemical technique except by tape-aided sectioning. By application of tape-aided sectioning, preparatory steps for sectioning (such as sample dehydration, paraffinization, and deparaffinization procedures, which would influence lipid content) can be safely omitted for the kernel and its sections. Sample grains can be directly embedded in the liquid paraffin, but the liquid paraffin cannot infiltrate into the grain because of moisture in the kernel. Other than the waxy paraffin slices, which are around the rice section and also stuck to the adhesive tape to repel the staining solution, only the grain section was stained. Figure 16.6 shows the resulting images of a virtually divided model for the three-dimensional isolated lipid


(a)

(b)

Figure 16.6 (a) An image of a virtually-divided three-dimensional visualizing model for the isolated lipid distribution and (b) its schematic form. In the three-dimensional model, black-stained parts such as the seed coat and the embryo are intentionally erased for better observation of the internal lipid distribution. Thus, this model represents areas below the seed coat of the rice kernel three-dimensionally. Although this model represents areas below the seed coat of the rice kernel and the sum of all the stained areas does not correlate quantitatively to the sum of all lipid-containing tissues, the display of the lipid distribution must be considered to have a qualitative character more than a quantitative value.

distribution, and its schematic form. The distinct lipid distribution at the divided plane can be shown. It is clear that the lipid tends to distribute at the dorsal side more prominently than at the ventral side in the sample kernel. Juliano (1972) and Takeoka et al. (1993) reported that lipid in the endosperm of rice existed most prominently in the cells of the aleurone layer, and its content was very small in the starch-storing tissue, which was located in the inner area of the rice kernel. By this visualization technique, the differences in lipid distribution in rice kernels of various cultivars, growth conditions, and post-harvest processing can be measured. Moreover, not only Sudan Black B but also other dyes can be applied for this technique. For example, for the visualization of differentiated lipid contents, which can be classified in fatty acids, neutral lipids, and so on, it has the potential to shed light on many phenomena, such as the mechanism of lipid autoxidation in a rice kernel. 3.3.3 Other imaging techniques

Atomic force microscopy (AFM) is a micro-imaging technique in which a sharp, probing tip is scanned over the surface of a sample. Interactions between the tip and the sample are translated into a three-dimensional image with resolution ranging from nanometers to micrometers. Using the AFM imaging technique, morphological features in the natural state and topographical information regarding biological samples, such as biological membranes, cell surface, and the molecular structure of various biological macromolecules, can be obtained. Dang and Copeland (2003) applied AFM imaging on the surface of cut grains of several rice varieties chosen on the basis of different amylase-amylopectin ratios and cooking properties. The angular starch structures (3–8 µm in size) were arranged in layers approximately 400 nm apart. The layers


represented the growth rings of starch granule formation, and the cross-striations in each layer corresponded to the blocklets of amorphous and crystalline regions within the starch granule. Such blocklets had an average size of 100 nm, and were proposed to comprise approximately 280 amylopectin side-chain clusters. The photoluminescence imaging technique, which is based on the spectral characteristics of visible light emitted from organic and inorganic compounds under UV irradiation, with video imaging and digital image processing, is suitable for quick and non-destructive quality control in various types of processing. Visible light photoluminescence from polished rice and some other starches was evaluated using a two-dimensional photoluminescence imaging technique in a quality control system for foods (Katsumata et al., 2005). Their visible light photoluminescence had a broad peak at a wavelength of 462 nm from starchy foods under illumination of ultraviolet light at 365 nm. Peak intensity of photoluminescence varies with the variety and the source of rice. The brightness over the photoluminescence image of rice of a single breed, from a single source, distributes according to a Gaussian distribution curve. The deviation of fitting result of brightness from the Gaussian distribution curve, which is estimated as χ2 value, and the correlation coefficient increased in rice specimens of various species of blended rice. Most grains, including rice, are composed of amylopectin, amylose, amino-acids, fatty-acids, and inorganic minerals, etc. Although the origin of the visible light luminescence from starchy foods is unidentified, lots of compounds emit visible light photoluminescence under UV irradiation. Thus, the relative contents of amylopectin and amylose, the concentration of amino acids and inorganic minerals such as Ca, Na, and K, may influence the photoluminescence intensity of rice. Because the quality of rice was influenced by these compounds, Katsumata and colleagues concluded that the photoluminescence imaging technique was potentially useful for quality evaluation of the rice. For example, the blended rice from different species could be detected using a two-dimensional photoluminescence imaging technique.

4 Quality evaluation of cooked rice 4.1 Water distribution Changes of the water distribution and the internal structure of a rice grain during cooking are closely related to the gelatinization characteristics, which influence the texture profiles of cooked rice. The gravimetric change in the rice grain during boiling was analyzed using a shell and core model developed by Suzuki et al. (1997). The model assumes that the gelatinization is much more rapid than the rate of water diffusion in a grain. This means that a partly boiled grain has an ungelatinized core covered by a gelatinized shell, which influences the eating quality of cooked rice. The core and the shell should therefore be measurable by the moisture profile in a grain. The geometrical changes in grains during cooking, which were followed by the kinetic study using various models, can be measured by on-line image acquisition. Ramesh (2001) inspected the swelling characteristics of whole rice grains under various temperatures using a digital image-analysis system on a real-time basis, and evaluated the cooking

Quality evaluation of cooked rice 391

kinetics of the whole rice grains. In order to determine the hot water hydration kinetics, he carried out the two-dimensional image analysis on basmati rice, which is a long-grain type, and could not apply a sphere mathematical model for moisture movement in rice. The projected area of rice grains was converted into swelling ratios for the comparison of the swelling at different temperatures. The reaction rate constant and the activation energy for the hot water hydration were obtained from the swelling data. The hydration data were further analyzed to generalize a polynomial equation correlating swelling ratios to the heating time and temperature. Ramesh also concluded that this helped the design of cooking equipment by providing a progressive increase in volume towards the discharge end to accommodate the swelling as cooking proceeds. Takeuchi et al. (1997a, 1997b) developed a real-time measurement technique for visualization of the moisture profiles in a rice grain during boiling by quick and onedimensional nuclear magnetic resonance (NMR) imaging with the adaptation of a multi-echo technique. Changes in the moisture distribution in a rice grain during boiling were evaluated by the proposed technique. The moisture content of the grain increased during boiling, and thus influenced gelatinization of the starch in the starch-stored cells. The moisture population map, which was represented by a virtual slice of the partly boiled grain, showed an asymmetrical progress of the moisture uptake in the grain. Therefore, they hypothesized that the water diffusion (moisture absorbance) in a grain was possibly restricted by cell-wall components of the starch-stored cells and the protein layers such as aleurone and subaleurone. They found, however, that cellwall materials had little effect on resisting moisture migration. Watanabe et al. (2001) proposed a non-Fickian diffusion mathematical model for water migration in rice grains during cooking based on the NMR-imaging technique. The migration of water is driven by the gradient of water demand, which is defined as the difference between the ceiling moisture content and the existing moisture content in the model. Their model was demonstrated to have potential for describing the anomalous characteristic features of water migration in a grain during cooking. The total limited water content within the rice grain was calculated and employed as an indicator of both the concentration and the distribution of water in the grain during cooking, observed by the NMR imaging technique (Kasai et al., 2005).

4.2 Grain-scale macro-structure Horigane et al. (1999) discovered the existence of hollows in a cooked rice grain, and proposed a mechanism to explain their formation, using NMR micro-imaging of protons (1 H). Their NMR micro-imaging techniques have mainly been limited to test-tube samples, allowing only a few grains to be analyzed at a time, although it is necessary to analyze multiple samples to ensure rapid, statistically sound conclusions. The samples were observed and analyzed by using two-dimensional images of longitudinal and transverse sections from three-dimensional NMR micro-images. Dark spots were found in the transverse sections and were surrounded by a peripheral layer of high proton-density. Such dark spots existed only within the grain, and caused no lacerations on the grain surface. The authors therefore hypothesized that a dark spot was due to either a low proton-density substance, or gases that appeared as a hollow. The


presence of gases in the hollow regions was only known to occur in cracks or fissures, and they confirmed, by a photomicrograph of the longitudinal section, that the dark spots were enclosed in hollow regions of the cooked grains. The hollows were related to structural changes that must have occurred during cooking, because no hollows were observed in uncooked grains except for cracks or fissures. They also found that the hollows appeared in the measurement of time-series images of the center layer in grains during cooking, which indicated changes in the internal structure and in water distribution. Accordingly, they concluded that the hollows originated from cracks or fissures of raw grain, and were caused by the sealing of such lacerations with gelatinized starch in the peripheral layer in combination with expansion of the grain during cooking. The hollows in a grain make the endosperm tissue less homogenous, and therefore influence the texture and structural properties of cooked rice. The hollows were also detected by other imaging techniques. Suzuki et al. (1999) applied X-ray imaging and a light transmittance photography techniques for the detection of hollows in a grain. They concluded that light transmittance photography was an effective and useful technique, although NMR imaging, the operation of which is very difficult, could provide more precise images and three-dimensional and quantitative measures of the depth and volume of hollows. Suzuki et al. (2002) also reported that hollow size, which was easily measured by the light transmittance method, was different in each cultivar. Hollow volumes were measured by the NMR micro-imaging technique (Horigane et al., 2000). The size, shape, and total volume of hollows for five cultivars with various amylose content were measured. Factors that influence hollow shape and volume should be related to either grain expansion or the gelatinization characteristics of the starchy endosperm. Endosperm amylose content is one such factor. However, they found that the volume increased during gelatinization, which was negatively related to the amylose content. Cracks and fissures in the grain, which are also important for the formation of hollows, occur prior to cooking in most cultivars due to soaking or changes in relative humidity. It seems unlikely that the differences in hollow formation among cultivars can be explained by the presence or absence of cracks and fissures. Thus, the volume should be measured, for example, the changes in hollow ratio during cooking, and calculated from the three-dimensional images constructed from serial slice images of the samples. In the research work of Horigane et al. (2000), the volume of hollows increased with grain volume and length during cooking below 100◦ C. Compared with this, the volume subsequently decreased during prolonged boiling. It was also assumed that there was a relationship between amylose content and hollow formation, based on their hypothetical model. However, it was concluded that there was no correlation between the final hollow volume and shape, and individual parameters such as flour gelatinization and amylose content. NMR micro-imaging was performed on a 7.1 T NMR spectrometer, and its settings resulted in a total acquisition time of 4.6 hours. Therefore, this approach inevitably involves long acquisition time and is not suitable for real-time observation of water transport in such a complex microstructure. Because a long scanning time and low spatial resolution lead to erroneous results, a short scan time and high spatial resolution are critical factors in the investigation of the cooking behavior of rice kernels.


Mohoric et al. (2004) proposed optimized three-dimensional NMR imaging with high spatial resolution based on the rapid acquisition relaxation enhanced (RARE) imaging method for the monitoring of cooking of a single rice kernel in real time. They aimed to achieve both high temporal and high spatial resolution in order to establish relationships between moisture content profiles, rice kernel microstructures, and the extent of gelatinization, and to develop the general pattern of moisture ingress. They used the three-dimensional RARE imaging sequence to record images of resolution of 128 × 32 × 16 voxels with a volume of 117 × 156 × 313 µm3 . An image was scanned in 64 seconds, and the images in time series spanned 30 minutes of the cooking process. Results were obtained from such real-time observation at high resolution, and the water uptake was determined by analysis of the magnetic resonance imaging (MRI). Results were compared with previous studies, and the general pattern of moisture ingress – i.e. the shape of moisture profiles and the actual facts – were generalized. Based on these results, a sophisticated model of water uptake in a three-dimensional substrate structure during different types of water diffusion, and the swelling of the substrate, can be developed for the simulation and interpretation of three-dimensional ingress patterns of moisture as observed by MRI imaging.

4.3 Cell-scale micro-structure Histological compound distributions, related to the physical, chemical and physicochemical properties of cooked rice, are also important in evaluating cooked rice qualities. In general, textural properties of cooked rice have great correlation with the morphological structures of a single kernel, and observation of histological structures (such as the compound distributions in the cooked grain) thus contributes to the quality inspection and evaluation of cooked rice. Microscopic techniques can be applied to individual grain kernels to identify the histological characteristics of cooked rice, and can also visualize structural details required for the evaluation of histological characteristics. To allow the digestibility of protein of rice, Bradbury et al. (1980) photographed the condition of histological components in a small segment of boiled rice kernel using the electron microscope. However, the rice kernel has inhomogeneous structures (Takeoka et al., 1993), and therefore not only small segments but also whole kernel sections are required for the evaluation of cooked rice grains. Although the frozen-sectioning method can be proposed for the collection of whole sections of a cooked kernel, it cannot produce quality sections because cooked rice kernels have poor physical properties for sectioning. Ogawa et al. (2003b) thus applied a combination method of tape-aided sectioning on a standard microtome and an autofluorescence visualization technique by microscope in the ultraviolet (UV) range to observe the histological properties, such as the location of phenolic cell wall materials, which were responsible for the autofluorescence produced using UV light (Fulcher, 1982). As a result, cell distributions, cell formations, and disruptions can be visualized. They also used scanning electron microscopy (SEM) as a complementary tool to fluorescence microscopy. Figure 16.7 shows a microscopy and an autofluorescent sample image of the same longitudinal section (20 µm thick) of a cooked rice kernel, which is stuck to adhesive tape. These images prove that quality sections for the cooked rice kernel can


(a)

(b) Figure 16.7 (a) Sample microscopy and (b) autofluorescent images of the same longitudinal section of cooked rice kernel stuck to a piece of adhesive tape. A simple image-processing algorithm for inverting the negative image and contrast enhancement to the autofluorescent image is carried out for better observation of visualized cell distributions. The section thickness is 20 µm; the magnification bar is 1 mm.

(a)

(b)

Figure 16.8 Magnifications of (a) the fringe and (b) the central area of the autofluorescent image in Figure 16.7. The magnification bar is 100 µm.

be obtained using the tape-sectioning method. In the autofluorescent image (Figure 16.7b), it can be seen that cell walls are destroyed at the outer area of kernels. Cell walls around the inner area are not destroyed in the non-void section (Figure 16.8). As demonstrated by the cell distributions and cell-wall formations at the various areas in the section, it was posited that cell walls tended to be damaged at the cells around the border between the rice and water during cooking, because internal areas of the non-void section had a clear distribution of cells similar to that of milled rice kernels. When rice is cooked, some compounds (such as carbohydrate and lipid) are dissolved into the cooking water, which gradually becomes concentrated and turns into a viscous liquid during boiling. This liquid becomes the membrane cover of the surface of cooked grain kernels in the final cooking stage, and is thus related to the eating quality


(a)

(b)

(c)

(d)

(e)

(f )

Figure 16.9 Images of microscopy for the histological sections of the compressed rice sample and their autofluorescent images focused in the void area. The compression ratios were 30% (a, d), 50% (b, e), 70% (c, f). These are perpendicular to the longitudinal direction. The magnification bar for the microscopy images is 1 mm, and for the autoflurescence images is 100 µm.

of rice (Hoshikawa, 1993a). Cell disruptions allow the dissolution of such internal compounds into the cooking water. Furthermore, cell disruptions are related to the texture. Because it is considered that the balance between cell disruptions and the dissolution of compounds into cooking water depends on the cooking process, differences in eating quality, which are caused by changes in cooking condition and recognized by experience, must be influenced by the histological and structural properties of the cooked grain kernels. Ogawa’s visualization technique can reveal the relationship between the histological, structural, and textural properties of cooked rice. Ogawa et al. (2006) studied the structural changes occurring in the cooked rice grain, after compression to a specific percentage, to show the relationship between texture and structure. The images are shown in Fig. 16.9. A compressed cooked rice grain was observed with the resistance force (Figure 16.10). The resistance force, which increased with the compression ratio, was linear up to 40 percent compression and was non-linearly increased when the compression ratio was above 50 percent (Figure 16.10). Microscopy samples were sectioned parallel to and viewed perpendicular to the direction of compression. The sections were collected at approximately the mid-point of each kernel in order to visualize the effects of compression. As the compression ratio increased, voided areas (empty or water-filled cavities) in the samples decreased and disappeared with higher compression ratios. At the initial phase of compression, voided areas offer little resistance to crushing. As compression increased, the dense and starchy material of the kernel absorbed the pressure and the cells began to be crushed. The differences in structure at various points of compression explain the linear behavior of the resistance force versus compression ratios up to 40 percent, and the non-linear behavior when the compression ratio increases beyond this level. In the uncompressed cooked kernel, relatively intact cells are found in the voided area. Surrounding the void area there are cells with disrupted cell walls and, therefore, free starch granules. Starch granules in the disrupted cells


10.0

Resistance force (N)

8.0

6.0

4.0

2.0

0.0

Figure 16.10

0

20

40 60 80 Compression ratio (%)

100

Averaged resistance force of cooked rice kernels against compression.

have already had access to water and become completely gelatinized with cooking, so the voids are sealed by gelatinized starch. Structural details of the voided areas show the way that voids and surrounding tissues change during compression. In the uncompressed kernels, the void is relatively narrow and pointed towards the lateral sides. Compression of the kernel at 30 percent causes the void to become wider in the center and perhaps split more towards the periphery of the kernel, whereas individual cells surrounding the void become only slightly distorted. Compressions of 50 percent and 70 percent, decreased the void volume, and distorted shapes of cells somewhat, affecting cell integrity. The effects of compression at 50 percent are less than those at 70 percent. Areas without voids are also compared structurally with those at the edge of the kernel. The cells of a cooked and uncompressed kernel are radially oriented as in a raw kernel, and the cells appear to be mostly intact. Compression at 70 percent causes cells to be more rounded in shape due to the effects of crushing, with surprisingly few areas where the cell walls have been torn. Cell walls are evidently capable of plastic deformation not only upon cooking but also with compression. The degree to which the kernel structure changed during the various compression tests, combined with the linear and then non-linear behavior of the resistance force versus the compression ratio, indicates that the voided areas and cell walls have an effect on texture. SEM images were also used to visualize changes of the compound formations contained in the rice grain during cooking or processing. Sandhya and Bhattacharya (1995) determined the relative rigidity/fragility of starch granules by SEM. They reported that the low-amylose rice starch showed total granule disintegration after 60 minutes of cooking at 95◦ C, but that high-amylose granules demonstrated only marginal disorganization in concentrated (12%) pastes. Waxy starch granules disintegrated even at 70◦ C. Apart from this, granules swelled without appreciable disruption and thus apparently more in low amylase starch than in high amylose starch in dilute (1%) pastes. Their results also indicated that the low-amylose starch granules were weak and fragile, and thus swelled and disintegrated easily. The high-amylose rice starch was relatively

References 397

strong and rigid, so it resisted swelling or disintegration. They therefore concluded that the relative rigidity/fragility of starch granules is key to differences in rice quality.

5 Conclusions The use of computer vision technology for raw and cooked rice quality evaluation, and results from this technique, have been summarized in this chapter, though there may be more related research such as near-infrared spectroscopic imaging and ultra-weak photon-emission imaging. Because computer vision technology is still progressing along with the development of hardware and software, lots of unidentified characteristics will be revealed. Such computerized techniques for rice quality evaluation should be followed not only by “real imaging techniques” but also by “virtual visualization techniques” – for example, hardness distribution with textural mechanics by parametric modeling.

References Atungulu G, NishiyamaY, Koide S (2003) Evaluation of condensation phenomena on grains by image analysis. Agricultural Engineering Journal, 12 (1&2), 65–78. Bechtel DB, Pomeranz Y (1977) Ultrastructure of the mature ungerminated rice (Oriza Sativa) caryopsis. The caryopsis coat and the aleurone cells. American Journal of Botany, 64 (8), 966–973. Bechtel DB, Pomeranz Y (1978a) Implications of the rice kernel structure in storage, marketing, and processing: a review. Journal of Food Science, 43 (5), 1538–1542. Bechtel DB, Pomeranz Y (1978b) Ultrastructure of the mature ungerminated rice (Oriza Sativa) caryopsis. The germ. American Journal of Botany, 65 (1), 75–85. Bechtel DB, Pomeranz Y (1978c) Ultrastructure of the mature ungerminated rice (Oriza Sativa) caryopsis. The starchy endosperm. American Journal of Botany, 65 (6), 684– 691. Bradbury JH, Collins JG, Pyliotis NA (1980) Methods of separation of major histological components of rice and characterization of their proteins by amino acid analysis. Cereal Chemistry, 57 (2), 133–137. Carter RM,YanY, Tomlins K (2006) Digital imaging based classification and authentication of granular food products. Measurement Science and Technology, 17, 235–240. Dalen GV (2004) Determination of the size distribution and percentage of broken kernels of rice using flatbed scanning and image analysis. Food Research International, 37, 51–58. Dang JMC, Copeland L (2003) Imaging rice grains using atomic force microscopy. Journal of Cereal Science, 37, 165–170. Food and Agriculture Organization of the United Nations (FAO) (2006) I. Production. FAO Rice Market Monitor, 9 (1), 3. Fulcher RG (1982) Fluorescence microscopy of cereals. Food Microstructure, 1, 167–175.


Furukawa S, Mizuma T, Kiyokawa Y, Masumura T, Tanaka K, Wakai Y (2003) Distribution of storage proteins in low-glutelin rice seed determined using fluorescent antibody. Journal of Bioscience and Bioengineering, 96 (5), 467–473. Goodman DE, Rao RM (1984) A new, rapid, interactive image analysis method for determining physical dimensions of milled rice kernels. Journal of Food Science, 49, 648–649. Horigane AK, Toyoshima H, Hemmi H, Engelaar WMHG, Okubo A, Nagata T (1999) Internal hollows in cooked rice grains (Oryza sativa cv. Koshihikari) observed by NMR micro imaging. Journal of Food Science, 64, 1–5. HoriganeAK, Engelaar WMHG,Toyoshima H, Ono H, Sasaki M, OkuboA, NagataT (2000) Differences in hollow volumes in cooked rice grain with various amylose contents as determined by NMR micro imaging. Journal of Food Science, 65, 408–412. Horigane AK, Engelaart WMHG, Maruyama S, Yoshida M, Okubo A, Nagata T (2001) Visualization of moisture distribution during development of rice caryopses (Oriza sativa L.) by nuclear magnetic resonance microimaging. Journal of Cereal Science, 33, 105–114. Hoshikawa K (1993a) Quality and shape of rice grains. In Science of the Rice Plant. Vol.1. Morphology (Matsuo T, Hoshikawa K, eds). Tokyo: Food and Agriculture Policy Research Center, pp. 377–412. Hoshikawa K (1993b) Rice seed, germination and seedlings. In Science of the Rice Plant. Vol. 1. Morphology (Matsuo T, Hoshikawa K, eds). Tokyo: Food and Agriculture Policy Research Center, pp. 91–109. Ishida N, Naito S, Kano H (2004) Loss of moisture from harvested rice seeds on MRI. Magnetic Resonance Imaging, 22, 871–875. Jia CC, Yang W, Siebenmorgen TJ, Bautista RC, Cnossen AG (2002) A study of rice fissuring by finite-element simulation of internal stress combined with high-speed microscopy imaging of fissure appearance. Transactions of the ASAE, 45 (3), 741–749. Juliano BO (1972) The rice caryopsis and its composition. In Rice, Chemistry and technology (Houston DF, ed). St Paul: American Association of Cereal Chemists, pp. 16–74. Kasai M, Lewis A, Marica F, Ayabe S, Hatae K, Fyfe CA (2005) NMR imaging investigation of rice cooking. Food Research International, 38, 403–410. Katsumata T, Suzuki T, Aizawa H, Matashige E, Komuro S, Morikawa T (2005) Nondestructive evaluation of rice using two-dimensional imaging of photoluminescence. Review of Scientific Instruments, 76 (7), 073702, 1–4. Kawamoto T, Shimizu M (1986) A method for preparing whole body sections suitable for autoradiographic, histological and histochemical studies. Stain Technology, 61, 169–183. Kennedy BM, Schelstraete M, Del Rosario AR (1974) Chemical, physical, and nutritional properties of high-protein flours and residual kernel from the overmilling of uncoated milled rice. I. Milling procedure and protein, fat, ash, amylose, and starch content. Cereal Chemistry, 51, 435–448. Lai FS, Zayas I, Pomeranz Y (1986) Application of pattern recognition techniques in the analysis of cereal grains. Cereal Chemistry, 63 (2), 168–172.

References 399

Lan Y, Fang Q, Kocher MF, Hanna MA (2002) Detection of fissures in rice grains using imaging enhancement. International Journal of Food Properties, 5 (1), 205–215. Levinthal C, Ware R (1972) Three dimensional reconstruction from serial sections. Nature, 236, 207–210. Lim MC, Lim KC, Abdullah MZ (2003) Rice moisture imaging using electromagnetic measurement technique. Transactions of the Institution of Chemical Engineers, Part C, 81 (3), 159–169. Mohoric A, Vergeldt F, Gerkema E, de Jager A, van Duynhoven J, van Dalen G, Van As H (2004) Magnetic resonance imaging of single rice kernels during cooking. Journal of Magnetic Resonance, 171, 157–162. Ogawa Y, Sugiyama J, Kuensting H, Ohtani T, Hagiwara S, Kokubo K, Kudoh K, Higuchi T (2000) Development of visualization technique for three-dimensional distribution of protein and starch in a brown rice grain using sequential stained sections. Food Science and Technology Research, 6 (3), 176–178. Ogawa Y, Sugiyama J, Kuensting H, Ohtani T, Hagiwara S, Liu XQ, Kokubo M, Yamamoto A, Kudoh K, Higuchi T (2001) Advanced technique for three-dimensional visualization of compound distributions in a rice kernel. Journal of Agricultural and Food Chemistry, 49 (2), 736–740. Ogawa Y, Kuensting H, Sugiyama J, Ohtani T, Liu XQ, Kokubo M, Kudoh K, Higuchi T (2002a) Structure of a rice grain represented by a new three-dimensional visualization technique. Journal of Cereal Science, 36 (1), 1–7. Ogawa Y, Kuensting H, Nakao H, Sugiyama J (2002b) Three-dimensional lipid distribution of a brown rice kernel. Journal of Food Science, 67 (7), 2596–2599. Ogawa Y, Orts WJ, Glenn GM, Wood DF (2003a) A simple method for studying whole sections of rice grain. Biotechnic & Histochemistry, 78 (5), 237–242. Ogawa Y, Glenn GM, Orts WJ, Wood DF (2003b) Histological structures of cooked rice grain. Journal of Agricultural and Food Chemistry, 51 (24), 7019–7023. Ogawa Y, Wood DF, Whitehand LC, Orts WJ, Glenn GM (2006) Compression deformation and structural relationships of medium grain cooked rice. Cereal Chemistry, 83 (6), 636–640. PalmgrenA (1954)Tape for microsectioning of very large, hard or brittle specimens. Nature, 174, 46. Ramesh MN (2001) An application of image analysis for the study of kinetics of hydration of milled rice in hot water. International Journal of Food Properties, 4 (2), 271–284. Sakai N, Yonekawa S, Matsuzaki A, Morishima H (1996) Two-dimensional image analysis of the shape of rice and its application to separating varieties. Journal of Food Engineering, 27, 397–407. Sandhya MR, Bhattacharya KR (1995) Microscopy of rice starch granules during cooking. Starch/Starke, 46 (9), 334–337. Shatadal P, Jayas DS, Bulley NR (1995a) Digital image analysis for software separation and classification of touching grain: I. disconnect algorithm. Transactions of the ASAE, 38 (2), 635–643. Shatadal P, Jayas DS, Bulley NR (1995b) Digital image analysis for software separation and classification of touching grain: II. classification. Transactions of the ASAE, 38 (2), 645–649.


Suzuki K, Aki M, Kubota K, Hosaka H (1997) Studies on the cooking rate equations of rice. Journal of Food Science, 42 (6), 1545–1548. Suzuki M, Horigane AK, Toyoshima H, Yan X, Okadome H, Nagata T (1999) Detection of internal hollows in cooked rice using a light transmittance method. Journal of Food Science, 64, 1027–1028. Suzuki M, Kimura T,Yamagishi K, Shinmoto H (2002) Discrimination of cooked mochiminori and koshihikari rice grains by observation of internal hollows using light transmittance photography. Food Science and Technology Research, 8 (1), 8–9. Takeoka Y, Shimizu M, Wada T (1993) Morphology and development of reproductive organs. In Science of the Rice Plant. Vol. 1. Morphology (Matsuo T, Hoshikawa K, eds). Tokyo: Food and Agriculture Policy Research Center, pp. 339–376. Takeuchi S, Fukuoka M, GomiY, Maeda M, Watanabe H (1997a)An application of magnetic resonance imaging to the real time measurement of the change of moisture profile in a rice grain during boiling. Journal of Food Engineering, 33, 181–192. Takeuchi S, Maeda M, Gomi Y, Fukuoka M, Watanabe H (1997b) The change of moisture distribution in a rice grain during boiling as observed by NMR imaging. Journal of Food Engineering, 33, 281–297. Wan YN (2002) Kernel handling performance of an automatic grain quality inspection system. Transactions of the ASAE, 45 (2), 369–377. Wan YN, Lin CM, Chiou JF (2002) Rice quality classification using an automatic grain quality inspection system. Transactions of the ASAE, 45 (2), 379–387. Wang YC, Chou JJ (2004) Automatic segmentation of touching rice kernels with an active contour model. Transactions of the ASAE, 47 (5), 1803–1811. Watanabe H, Fukuoka M, Tomiya A, Mihori T (2001) A new-Fickian diffusion model for water migration in starchy food during cooking. Journal of Food Engineering, 49, 1–6. Watson CA, Dikeman E (1977) Structure of the rice grain shown by scanning electron microscopy. Cereal Chemistry, 54 (1), 120–130. Yadav BK, Jindal VK (2001) Monitoring milling quality of rice by image analysis. Computers and Electronics in Agriculture, 33, 19–33. Yasumatsu K, Moritaka S (1964) Fatty acid compositions of rice lipid and their changes during storage. Agricultural and Biological Chemistry, 28 (5), 257–264. Yiu SH (1993) Food microscopy and the nutritional quality of cereal foods. Food Structure, 12, 123–133.

Quality Evaluation of Corn/Maize Stephen J. Symons and Muhammad A. Shahin Grain Research Laboratory, Winnipeg, Manitoba, Canada, R3C 3G8

1 Introduction There has been a long-term consistent effort by scientists to move away from subjective evaluation of seed properties and towards objective inspection. Even prior to computer-based quantification, simple seed parameters were being quantified using numerical techniques – for example, oat kernels were placed on size grids and measurements were determined by estimating the proportion of squares covered by the kernel (Baum and Thompson, 1976). The earliest attempts at quantifying quality parameters in cereals using automated or computer-based assessment were in the application of imaging to assess gliadin electrophoregrams. Lookhart et al. (1983) used a computer application to compare the gliadin banding pattern from an “unknown” wheat variety with band patterns obtained from known varieties. By doing so, they could predict the membership of the unknown kernel to a known variety grouping. This was an indirect approach to implementing computer analysis, for the acquisition of the electrophoretic patterns was indirect via a photographic reversal negative that was subsequently scanned by a spectrodensitometer. Sapirstein and Bushuk (1985a) adapted the imaging technique for electrophoretic analysis to use a digitizer to acquire information from electrophoregrams for analysis. This technique, along with their modified band relative mobility algorithms, improved the analysis of electrophoregrams for wheat cultivar identification (Sapirstein and Bushuk, 1985b). The application of machine vision systems for the measurement of seed characteristics started with relatively simple measurements. The concept of simple counting was applied in the earliest imaging systems, some of which required the user manually to trace the outline of their object of interest, while others incorporated the ability to segment the object from the background (e.g. the IBAS system, Kontron Electronics, Eching, Germany). The application of automated imaging techniques to assess attributes of the cereal grains has been reported for almost 30 years. Simple measurements of seed area, length, and either width or height were used to feed discriminant Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


17

402 Quality Evaluation of Corn/Maize

models to identify different UK wheat varieties (Travis and Draper, 1985; Keefe and Draper, 1986), and different soft red US wheat varieties (Zayas et al., 1985).

1.1 Whole seed analysis for type Using a Quantimet 720 image analyzer, Zayas et al. (1985) were able to directly digitize wheat seeds using a vidicon-type tube camera. Following daily corrections for background effects and shading, seed morphology parameters were calculated. Using a set of directly determined parameters, nine derived parameters were used to separate kernels of two wheat types, namely Arthur and Arkan, using a canonical analysis technique. There was a high degree of identification of each wheat type. A Quantimet 10 image analysis system was used by Keefe and Draper (1986) to measure several parameters from the side perspective of wheat kernels. Again, a camera, connected to the computer using an analog-to-digital converter, was used to directly capture images of the seeds (Draper and Travis, 1984; Travis and Draper, 1985). A total of 16 parameters were used: 10 were directly measured, while the other 6 were derived. An analytical approach described by Almeida and Bisby (1984) was used for comparing sets of variables. The method presented by Keefe and Draper (1986) was able to identify whether seed samples were derived from the same seed lot, or were of the same variety. Analysis of cereal grains required the exact manual positioning and location of the kernel, since each measurement was orientation specific (Zayas et al., 1985; Keefe and Draper, 1986; Symons and Fulcher, 1987). These systems also separated image capture, image processing, and data analysis into stand-alone steps. In the next evolution of the technology, image capture, image processing, and image and data storage were integrated by linking two computer systems (Zayas et al., 1986). Keefe and Draper (1988) reported a system for the automated location of the sample under the camera, allowing multiple views. In their instrument, the camera moved in both X and Y dimensions along a gantry that facilitated scanning a large number of samples. While the initial application of imaging within cereals focused on the separation of different sub-types or cultivars within a cereal type, such as wheat, corn or barley, this was eventually expanded to the determination of membership of seeds from several different seeds types. Lai et al. (1986) used discriminant analysis techniques to classify and predict the membership of seven grain types, namely corn, soybean, grain sorghum, white rice, brown rice, wheat, and barley. Figure 17.1 illustrates several of the grain types that are commonly used in imaging studies. Corn, having unique shape and color (translucency) characteristics in addition to being the largest seed type in the study, gave 100 percent accurate identification. Data models were also created that characterized each of the seed types. However, caution must be exercised when dealing with the characterization of biological products such as seeds. While seed characteristics are highly influenced by the genetics of the parent, their appearance is also subject to substantial influence from the growing environment. In a study of kernels of wheat grown in eastern Canada, variations in kernel characteristics for a single variety grown at multiple locations were as large as the differences found between different varieties grown in one location (Symons, unpublished). This implies that large sample sets

Introduction 403

(a)

(b)

(c)

(d)

(e) Figure 17.1 Examples of various commodities imaged: (a) corn; (b) soybean; (c) sorghum; (d) red spring wheat; (e) malting barley. © Canadian Grain Commission.

exhibiting the total variability anticipated for a seed type or variety are required to create robust data models.

1.2 Internal seed characteristics The functional properties of cereal grains as they relate to processing or milling are dependent upon more than just the external morphometry of the seed. Indeed, functional properties are significantly influenced by the internal relationship of tissues during seed formation and maturation. The internal characteristics of wheat seeds of European wheats were found to be related to the milling properties (Evers and Withey, 1989), although the seed preparation methodology was found to be less than optimal and


would be difficult to adapt to a routine fast analysis. The internal tissue relationships for Canadian wheats were related to variety (Symons and Fulcher, 1988a), suggesting that measurement of these properties would be beneficial to the overall determination of seed quality and perhaps variety. Certainly, the inclusion of these properties into a model to classify varieties had a significant benefit.

1.3 Relating seed morphometry to quality The rationale for the determination of cereal grain variety is that different varieties exhibit differing quality characteristics. For oats, a distinct relationship between oat milling yield and kernel weight was reported (Symons and Fulcher, 1988b). A strong relationship between oat kernel area and kernel mass was established (Symons and Fulcher, 1988c), allowing the potential application of imaging measurements to predict oat milling yield. This solution, however, is not simple, since there is a wide diversity in oat kernel weight within a sample due to kernel location in the oat plant head. Furthermore, the growing location also has a significant effect on kernel characteristics (Symons and Fulcher, 1988c). A model showing the successful prediction of oat milling quality has yet to appear in the literature. One of the perceptually easier problems to tackle in cereals is the determination of kernel vitreousity. Sapirstein and Bushuk (1989) showed that durum wheat has a high transmittance profile compared to hard red spring wheat, rye, barley and oats. They also demonstrated that kernels of durum wheat with differing degrees of vitreousness had substantially differing light transmittance profiles, and thus there was a potential for using imaging methodology to predict durum wheat hard vitreous kernel (HVK) scores. The proportion of HVK in a sample is an internationally recognized specification determining the value of durum wheat, and is a quality factor that is currently visually assessed, so remains a prime candidate for machine evaluation. Seed translucency of durum wheat kernels (Figure 17.2) was able to predict HVK scores in a set of five prepared samples in the HVK range of 20–100 percent, and was accurate for the prediction of commercial cargo shipments of Canadian durum wheat (CWAD) to within 5 percent (Symons et al., 2003). When measurements from reflective images were combined with those from light transmittance images, a high degree of consistency was found (Xie et al., 2004). In this work, the machine vision system gave a standard deviation of 1 percent compared to trained inspectors with 3.5 percent. Xie et al. (2004) also confirmed the observations of Symons et al. (2003) that mottled (piebald) kernels and bleached (weathered) kernels were difficult to classify accurately, since these characteristics are difficult to image and therefore to model numerically.

1.4 Assessing seed quality indirectly Grain hardness is an important quality characteristic, as it directly affects the milling characteristics and the level of starch damage that may occur during milling. Hardness is not easily determined from whole grain morphometic analysis, although, as discussed in section 1.2, it is related to grain vitreousness, and HVK determination may predict some milling properties of the grain. Zayas et al. (1994) approached the

Introduction 405

(a)

(b)

(c) Figure 17.2 (a) Vitreous Canadian durum wheat; (b) non-vitreous Canadian durum wheat; (c) visual appearance of the two quality types – transmitted light or backlight images of the two quality types showing translucent vitreous kernels and opaque starchy kernels. © Canadian Grain Commission.

problem of determining grain hardness by characterizing the isolated starch granules from samples of both US hard red winter (HRW) and soft red winter (SRW) wheats. Using starch granule size and aspect ratio measured by imaging, full segregation of the soft and hard wheats was demonstrated. This separation was confirmed using standard near-infrared analysis to determine sample hardness. There has been no clear demonstration that the milling properties of common wheats can be determined from whole grain morphometery, yet the characteristics of flour streams in the mill and flour refinement (Symons and Dexter, 1991), which are related to both value and functional properties, can be determined by imaging. The outer seed-coat layers have characteristic fluorescent properties for both the pericarp (Symons and Dexter, 1992, 1996) and the aleurone (Symons and Dexter, 1993), which relate to traditional flour quality determinants (Kent Jones Color, L∗ or Ash content) used for describing flour refinement.


Again, while there is no report of an imaging system that characterizes the milling properties of durum wheat as related to semolina quality, imaging methods have been developed that can characterize the semolina (Symons et al., 1996), particularly the speckiness, which directly relates to milling quality and market value.

1.5 Adding color into the analysis Color is a visual property of agricultural products, since it is correlated in many cases to other physical, chemical, and sensorial indicators of product quality (Mendoza et al., 2006). The analysis of images obtained using a color camera is equivalent to the analysis of three monochrome images obtained through wide-band red (R), green (G), and blue (B) filters. A difference in reflectance of all three-color planes was found between types of western Canadian wheats (Neuman et al., 1989a). These color differences were used to identify and classify the wheat classes that differed in color (Neuman et al., 1989b). The Canadian grain grading system uses differences in wheat kernel size, shape, and color to distinguish the wheat class (Figure 17.3), which in turn relates to functional processing characteristics. For seeds, of which color formed the primary determinant of quality, color measurements from images have a significant role in predicting quality. Varieties of five western Canadian lentils were separated into color classes using colorimaging measurement. When combined with size determination, all five varieties were determined with an accuracy of 99 percent (Shahin and Symons, 2005). Classification was performed using a neural network model. Similarly, using a back-propagation type neural network, five rice cultivars were classified and identified with a high degree of accuracy when color features were incorporated into the classification model (Liu et al., 2005).

1.6 The analysis is only as good as the sample When there was a high degree of sample variability, such as in durum wheat rail-car samples, there was less agreement between trained inspectors and a machine vision system (Symons et al., 2003). This result has been experienced multiple times by the current authors in their research program (unpublished). Seed samples containing clearly defined groups remain relatively easily classified using measurements derived from images. However, commercial samples, which may be somewhat heterogeneous in composition, pose challenges in obtaining a high degree of classification accuracy. The use of computer vision can give a measurement error of less than 0.1 mm for samples of no less than 300 kernels. This could be used to characterize high-quality commercial grain shipments (Sapirstein and Kohler, 1999). However, it was found that as the overall sample quality declined, the sample size required to maintain this accuracy of analysis increased, reflecting a lower uniformity in kernel morphometry.

1.7 Integration and automation of analysis Early imaging instruments usually separated the image capture procedure from image data analysis. More complex instruments (Symons and Dexter, 1991; Zayas et al., 1994)

Introduction 407

Prairie Spring (Red)

Red Spring

Prairie Spring (White)

Extra Strong Red Spring

Soft White Spring

Durum

Red Winter

Figure 17.3 Canadian wheat classes differing in kernel size, shape, and color. © Canadian Grain Commission.

integrated both the image capture and image processing into a single instrumental process, although data analysis remained a distinctly separate process. This independence of each step in the imaging process is acceptable in a research environment; however, it is not acceptable if machine vision is to be deployed as an applied technology in the industry for grain quality analysis. Research systems typically used hand-positioned grain kernels to ensure consistency of imaging measurements. However, automation dictates that this would not be possible and that alternative mechanisms need to be sought. To this end, investigations of disconnect algorithms for the separation of touching wheat kernels was undertaken (Shatadal et al., 1995a). This work demonstrated that in samples presented as a monolayer to an imaging system, better than 80 percent of touching instances could be separated. The effect of the disconnect algorithm


applied to touching kernels was minimal for measured kernel features, although the area of oats and the radius ratio of barley were adversely affected. Wheat features remained unaffected by this technique (Shatadal et al., 1995b). Touching seeds of Canadian pulse crops were singulated in multi-layer samples. The size distributions of seeds as determined by image analysis matched closely with the size distributions determined by sieving, the standard industrial sizing method (Shahin and Symons, 2005). These studies indicate that there is potential for automation of seed delivery to imaging systems, since the resolution of touching seeds is possible using imaging techniques. The integration of all steps, from image analysis to data analysis and information delivery, has only recently been reported for grain analysis. Shahin and Symons (2001) report an integrated analysis system for Canadian lentils, while DuPont Acurum (www.acurum.com) report an integrated instrumental system for cereal grains analysis.

2 Corn The USA is a major – possibly the largest – corn grower and international exporter. Corn (Zea mays L.) is grown as a food, feed, and industrial feedstock. The commercial value of corn is based on the seed quality, which in turn determines the end use of the product. End use of corn varies widely. Approximately 80 percent is consumed as animal feed for meat, poultry, and milk production, while the remaining 20 percent is used in a variety of industrial processes for production of oil, starches, high-fructose corn sweetener, ethanol, cereals, and other food products (Hurburgh, 1989). On average, corn kernels consists of 71 percent starch, 9 percent protein, and 4 percent oil on a dry weight basis; however, genetic background and environment conditions cause significant variations in constituent contents (Hurburgh, 1989).

2.1 Use of corn Historically, a major portion of US corn has been grown for animal feed. However, the fastest growing use of corn today is for food and industrial use. Most of the growth in industrial use of corn has been in the area of wet milling, which currently accounts for 75 percent of processed corn (Eckhoff, 1992). Corn wet-milling is an industrial process that separates the corn kernel into its starch, protein, germ, and fiber fractions. Growth in the wet-milling industry was especially rapid during the 1970s because of breakthroughs in the production and subsequent use of high-fructose corn syrup (Leath and Hill, 1987). Dry milling is the process that separates corn into endosperm, germ, and fiber fractions. Dry milling has seen some limited growth, primarily because of increased consumption of breakfast foods and other dry-milling products (Leath and Hill, 1987).

2.2 Corn grading For grading purposes, corn is classed as yellow, white, or mixed. Samples of yellow and white corn containing less than 95 percent of one class are designated Mixed. According

Corn 409

to Canadian standards, primary grade determinants are minimum test weight, degree of soundness (size uniformity), damaged kernels (heated, mold contamination, etc.), cracked corn, and foreign material (OGGG). Corn is graded without reference to variety. The class forms part of the grade name – e.g. Corn, Sample CW Yellow Account Heated. Corn quality factors are important for both wet and dry milling. For wet milling, it is important to ensure that the corn kernels have not been dried at temperatures high enough to cause protein denaturation or starch gelatinization. Stress crack percentages are used as an indirect test for these conditions (Rausch et al., 1997). Stress cracks below 20 percent will enable a high starch recovery from wet milled corn. The primary factor needed by dry millers is a hard endosperm, which is used to produce large, flaking grits (Paulsen et al., 1996). Many overseas dry millers prefer true densities in the range of 1.25–1.28 g/cm3 . High-density kernels are usually more difficult to steep adequately, resulting in lower starch recovery. The secondary factor needed by dry millers is low stress cracks, preferably below 20 percent. Artificial drying of corn can cause two types of damage. Rapid drying causes brittleness. This is the most prevalent damage, and is manifested in the form of stress cracks leading to breakage. Stress cracks directly affect the ability of millers to salvage intact endosperms, and generally reduce the number of large, premium grits produced in dry milling. Stress cracks also contribute to the breakage in corn during its handling. Scorching and discoloration of corn characterize damage caused by overheating. This indirectly contributes to the brittleness of the dried grain. Heat damage caused by excessive drying temperatures not only results in physical damage to the kernel that affects milling properties, but also causes undesirable chemical changes that make starch and gluten separation difficult in wet milling. Corn is usually harvested at moisture contents of between 18 and 25 percent. However, periodic early frosts or wet fall weather coupled with a producers’ desire for timely harvest may necessitate harvest at a higher moisture level followed by hightemperature drying. According to a US Grain Council producer survey (Anonymous, 2001), more than 50 percent of on-farm corn-drying takes place at temperatures well above the starch gelatinization temperature (>70◦ C). Artificial drying at high temperatures is known to induce stress cracks and reduce the germ quality, starch recovery, starch quality, flaking grit yield, and storage life of corn. This can result in poor characteristics for wet milling, dry milling, handling, and storage (Freeman, 1973; Brooker et al., 1974). Excessive stress cracking increases the amount of fines and broken corn during handling, which in turn increases susceptibility to mold and insect damage during storage. In the dry-milling industry, high-temperature drying reduces grit yields because of increased stress cracks, and reduces germ recovery and grit quality due to poorer germ–endosperm separation (Paulsen and Hill, 1985). In the wet-milling industry, high-temperature drying makes corn difficult to steep by altering the characteristics of the protein matrix in the endosperm and increasing the time for adequate steeping. Inadequate steeping results in poor starch–gluten separation, reducing starch yield and quality while increasing the yield and decreasing the purity of lower-valued protein products (Watson and Hirata, 1962).


3 Machine vision determination of corn quality Quality parameters for corn kernels have been determined using machine vision in both the densitometric and spatial domains. Machine vision sensing has been used to develop methods for detecting and quantifying physical properties of corn in order to develop the basis for on-line grain quality evaluation as a tool for assessing grain quality and end-use of the grain. Machine vision systems have been developed for assessing color, size, shape, breakage, stress crack, hardness, fungal contamination, and seed viability in corn, as described below.

3.1 Color The color of foods is an important quality factor that greatly influences consumer acceptance (Mendoza et al., 2006). Processors want clean, brightly colored corn for food products. For grading purposes, trained inspectors visually observe the color of kernels to determine the class of corn (white or yellow). However, the color of corn kernels can vary considerably from white to yellow, orange, red, purple, and brown (Watson, 1987; Figure 17.4). White food corn hybrids vary from a pale, dull white to a gray off-white appearance, while yellow hybrids can range from a light yellow to a dark reddish yellow color. Bright, clean yellow and white kernels are desired for food corn. Most of the methods used in the field are subjective measurements. Objective color measurement methods are important for processors and breeders developing new corn varieties. In laboratories, corn color is typically measured using a colorimeter or spectrometer that records the color values in the CIE Lab color space and its derivatives (CuevasRodrigues et al., 2004). Floyd et al. (1995) used a colorimeter to measure L, a, b color components in white and yellow corn samples. They observed low correlations (r ≤ 0.53) between the instrumental measurements and the color grade by a trained inspector. Differences in endosperm hardness, kernel thickness, and pericarp gloss between cultivars with the same visual color ratings contributed to the low correlation values. Liao et al. (1992a) used machine vision to discriminate corn kernels based on RGB color values. They reported that the values of the red (R) component of the corn kernel images were higher than the values of the green (G) component. Later studies found that the kernel color could be quickly determined after deriving HSI (Hue, Saturation, Intensity) from the RGB input image (Liao et al., 1994). The largest difference between white and yellow maize varieties was found in the intensity component of the image, while the blue component of the RGB image provided the greatest separation between white and yellow corn kernels. In each case the standard deviation was low, allowing for clear separation of the corn types (Liao et al., 1994). The effectiveness of color-image analysis and classification techniques depends on the constancy of the scene illumination, whereas scene illumination often changes over time. Ng et al. (1998a) presented a calibration method to improve color-image classification for changing illumination. They used a gray reference plate to capture color changes due to changes in illumination. The extent of the color changes in each of the RGB channels was calculated based on an equation derived from the spectral reflectance model. These values formed a transformation matrix to transform the image

Machine vision determination of corn quality 411

(a)

(b) Figure 17.4 (a) Multicolored varieties of corn. © This digital image was created by Sam Fentress, 25 September 2005. This image is dual-licensed under the GNU Free Documentation License, Version 1.2 or later, and the Creative Commons Attribution Share-Alike license version 2.0. Attribution is required. Please direct any questions to User talk:Asbestos. (Sam Fentress). (b) Exotic varieties of corn with different kernel color. This public domain image is from Wikipedia, the free encyclopedia (http://en. wikipedia.org/wiki/Image:GEM_corn.jpg).

RGB values to compensate for the color changes. The color-corrected RGB values were shown to be within four gray levels of the laboratory measurements for a 1-V change in the lamp voltage. Liu and Paulsen (2000) successfully used machine vision to quantify whiteness in corn samples with large color variations.


3.2 Size and shape Seed corn is marketed by kernel size, making kernel size distribution a very important characteristic for the seed corn industry. The ability of a mechanical planter to meter seeds at a consistent spacing improves with uniformly sized seed, which in turn affects the yield. An ear of corn contains a large number of kernels, each with a slightly varying physical size and shape reflecting its position on the ear – seeds on the tip of an ear tend to be small and round, seeds in the middle of an ear tend to be flat, and seeds on the bottom of an ear tend to be large and triangular. Machine vision systems have been developed to determine kernel size and shape characteristics. Liao et al. (1992b) identified corn kernel profiles from morphological features, including curvatures along the kernel perimeter, symmetry ratios along the major axis, aspect ratios, roundness ratios, and pixel area. Ni et al. (1997a) developed a machine vision system to identify different types of crown-end shapes of corn kernels. Corn kernels were classified as convex or dent, based on their crown-end shape (Figure 17.5). Dent corn kernels were further classified into smooth dent or non-smooth dent kernels. This system provided an average accuracy of approximately 87 percent compared with human inspection. Winter et al. (1997) measured morphological features of popping corn kernels using image analysis. This information along with pixel value statistics was used to predict the “popability” of popcorn using a neural network. Ni et al. (1998) used a mirror to capture both top and side views of corn kernels, to determine kernel length, width, and projected area for size classification. This size-grading vision system performed with a high degree of accuracy (74–90 percent) when compared with mechanical sieving. Steenhoek and Precetti (2000) developed an image-analysis system for the classification of corn kernels according to size categories. Kernels were classified into 16 size categories based on the degree of roundness and flatness determined by morphological

Dent

Convex

Figure 17.5

Gray-scale and binary images showing different shapes of corn kernel crown.


features measured from seed images. Neural network classification accuracies for round and flat kernels were 96 percent and 80 percent, respectively. These reports demonstrate that there is considerable potential for a machine vision system for corn segregation.

3.3 Breakage Current practices of harvesting high-moisture corn introduce substantial mechanical damage to kernels, which is further aggravated by subsequent handling and transportation operations. It is estimated that on-farm mechanical damage to corn kernels ranges from 20 to 80 percent (Pierce et al., 1991). Such damage includes kernels that have hairline cracks, as well as those that are broken, chipped, or crushed. Damaged corn is more difficult to aerate, and has a shorter storage life than undamaged corn. Mechanical damage is frequently measured in laboratories through visual inspection, which is subjective, tedious, and time-consuming. Large-scale measurement of corn damage for the grain trade is not practical unless the process is fully automated. Machine vision systems have been developed for measuring corn kernel breakage, with promising results (Ding et al., 1990; Zayas et al., 1990). Liao et al. (1993) developed a machine vision system to measure corn kernel breakage based on the kernel shape profile. Diffused reflected light illuminated the single kernels for image capture. A neural network classifier achieved high classification accuracy; 99 percent for whole flat kernels, 96 percent for broken flat kernels, 91 percent for whole round kernels, and 95 percent for broken round kernels. Liao et al. (1994) further improved this system by including a Fourier profile of the kernel. The improved system had an accuracy of 95 percent in identifying whole kernels as being whole, and 96 percent accuracy for identifying broken kernels as being broken. Parameters such as projected area, width, and height of the kernel were determined, in addition to Fourier coefficients using an FFT (Fast Fourier Transform). Ni et al. (1997b) designed and built a prototype machine vision system for automatically inspecting corn kernels. They used a strobe light to eliminate image blur due to the motion of corn kernels. Kernel singulation and the synchronization of strobe firing with the image acquisition were achieved by using optical sensors. The control circuitry was designed to enable synchronization of strobe firing with the vertical blanking period of the camera. Corn kernels of random orientation were inspected for whole versus broken percentages, and on-line tests had successful classification rates of 91 percent and 94 percent for whole and broken kernels, respectively. Ng et al. (1998b) developed machine vision algorithms for measuring the level of corn kernel mechanical damage as a percentage of the kernel area. Before imaging, corn samples were dyed with a 0.1% Fast Green FCF dye solution to facilitate the detection of damaged areas. Mechanical damage was determined by extracting from the kernel images the damaged area stained by the green dye as a percentage of the projected kernel area. The vision system demonstrated high accuracy and consistency. Standard deviation for machine measurements was less than 5 percent of the mean value, which is substantially smaller than for other damage-measurement methods. This method is, however, limited in that it introduces a dye into the grain product.


3.4 Stress cracks Internal damage in corn appears in the form of stress cracks in the endosperm (Thomson and Foster, 1963). These cracks have traditionally been evaluated by candling and visual assessment. Candling is time-consuming and inconsistent, due to fatigue of the human eye. Gunasekaran et al. (1985) investigated the size characteristics of stress cracks using electron microscopy. They observed that a typical stress crack is about 53 µm in width and half the kernel in depth. Stress cracks originate at the inner core of the floury endosperm, and propagate rapidly outwards along the boundary of starch granules. Many cracks do not advance as far as the pericarp layer. Reflected laser optical imaging failed to provide sufficient light reflectance differences required for detecting stress cracks (Gunasekaran et al., 1986) whereas backlighting images provided high contrast between the stress crack and the rest of the kernel (Gunasekaran et al., 1987). Image-processing algorithms detected the cracks in the form of lines or streaks with an accuracy of 90 percent. The general principles of backlighting for transmittance imaging are illustrated in Figure 17.6. Backlight imaging reveals useful information about the internal structure of grain samples by generally eliminating details from the surface and providing high contrast for edge detection. Reid et al. (1991) developed a computer vision system to automate the detection of stress cracks in corn kernels. They used a combination of reflected (diffused) as well as transmitted light for imaging single kernels. Edge detection followed by Hough transform was used to detect stress cracks as line features. This system detected stress cracks with an accuracy approaching 92 percent in comparison to human inspection with candling. Han et al. (1996) used Fourier transform image features for the inspection of stress cracks. The proposed frequency domain classification method achieved an average success ratio of 96 percent.

Light sensor

Sample

Light Figure 17.6

A generalized schematic of backlight imaging set-up. © Canadian Grain Commission.


3.5 Heat damage Milling processes are designed to separate kernels efficiently into their respective components. Corn that has been heated in the presence of moisture has difficulty during the starch–gluten separation phase in wet milling. These problems result from protein denaturation or starch gelatinization. Heat damage was visualized using tetrazolium dye, which turns pink in living embryos but shows no color in dead tissues (Xie and Paulsen, 2001). Dehydrogenase enzymes involved in respiration react with the tetrazolium, resulting in an insoluble, red formozan color in living cells. Non-living cells retain their natural color. Machine vision images of kernels that were heat treated at 60◦ C for 3 and 9 hours respectively were compared to check samples that were not heated. The unheated kernels had a tetrazolium reaction resulting in a bright red stain. Kernels heated for 3 hours had a purplish color, indicating onset of damage; while the kernels heated for 9 hours did not stain, indicating a totally dead germ (Paulsen and Hill, 1985; Litchfield and Shove, 1989).

3.6 Mold and fungal contamination Ng et al. (1998b) developed machine vision algorithms for measuring corn kernel mold damage using color images of corn kernels illuminated with diffused reflectance lighting. Mold damage was determined in terms of percentage of total projected kernel area by isolating the moldy area on kernel images. A feed-forward neural network was developed to classify mold and non-mold pixels, based on pixel RGB values. The system measurements were highly accurate and consistent, with a standard deviation of less than 5 percent of the mean value. Steenhoek et al. (2001) presented a method for clustering pixel color information to segment features within corn kernel images. Features for the blue-eye mold, germ damage, and starch were identified with a probabilistic neural network based on pixel RGB values. Accuracy of the network predictions on a validation set approached 95 percent. Aflatoxins are poisons produced by the fungus Aspergillus flavus after it infects agricultural commodities, such as corn. Aflatoxin-contaminated corn is dangerous when consumed by animals or human beings, and therefore is an undesirable characteristic for any corn that is going for feed or human consumption. The ability to detect A. flavus and its toxic metabolite, aflatoxin, is important for health and safety reasons. The ability to detect and measure fungal growth and aflatoxin contamination of corn could contribute significantly towards the separation of contaminated kernels from healthy kernels. Dicrispino et al. (2005) have explored the use of hyperspectral imaging to detect mycotoxin-producing fungi in grain products. Experiments were performed on A. flavus cultures growing over an 8-day time period to see if the spectral image of the fungus changed during growth. Results indicate that hyperspectral imaging technology can identify spectral differences associated with growth changes over time. Further experiments may lead to this technology being used to rapidly and accurately detect/measure Aspergillus flavus infection/aflatoxin contamination of corn without destruction of healthy grain. This could provide a useful tool for both growers and


buyers in the corn industry, that could enhance protection of food and feed as well as increase profits. The bright greenish-yellow (BGY) presumptive test is widely used by government agencies as a quick test for monitoring corn aflatoxin to identify lots that should be tested further. The test is based on the association of the BGY fluorescence in corn under ultraviolet light (365 nm) with invasion by the molds that produce aflatoxin. Shotwell and Hesseltine (1981) examined corn samples under ultraviolet light (365 nm) for the bright greenish-yellow (BGY) fluorescence associated with aflatoxin-producing fungi. They concluded that the BGY test could be carried out equally well by using the black light viewer on whole-kernel corn or by inspecting a stream of coarsely ground corn under ultraviolet light (365 nm). A count of 1 BGY particle per kg of corn appeared to be an indication that the sample should be tested for aflatoxin by chemical means. The higher the BGY count in a corn sample, the more likely it is to contain aflatoxin in levels equal to or exceeding the tolerance limit of 20 ng/g. Near-infrared spectra, X-ray images, color images, near-infrared images, and physical properties of single corn kernels were studied to determine whether combinations of these measurements could distinguish fungal-infected kernels from non-infested kernels (Pearson et al., 2006). Kernels used in this study were inoculated in the field with eight different fungi: Acremonium zeae, Aspergillus flavus, Aspergillus niger, Diplodia maydis, Fusarium graminearum, Fusarium verticillioides, Penicillium spp., and Trichoderma viride. Results indicate that kernels infected with Acremonium zeae and Penicillium spp. were difficult to distinguish from non-infested kernels, while all the other severely infected kernels could be distinguished with greater than 91 percent accuracy. A neural network was also trained to identify infecting mold species with good accuracy, based on the near-infrared spectrum. These results indicate that this technology can potentially be used to separate fungal infected corn using a high-speed sorter, and to automatically and rapidly identify the fungal species of infested corn kernels. This will be of assistance to breeders developing fungal-resistant hybrids, as well as mycologists studying fungal-infected corn. Pearson and Wicklow (2006) used a high-speed single-kernel sorter to remove mycotoxins from corn. It was found that using spectral absorbance at 750 nm and 1200 nm could distinguish kernels with aflatoxin contamination greater than 100 ppb from kernels with no detectable aflatoxin, with over 98 percent accuracy. When these two spectral bands were applied to sorting corn at high speeds, reductions in aflatoxin averaged 82 percent for corn samples with an initial level of aflatoxin over 10 ppb. Most of the aflatoxin is removed by rejecting approximately 5 percent of the grain. Fumonisin is also removed along with aflatoxin during sorting. The sorter reduced fumonisin by an average of 88 percent for all samples. This technology will help insure the safety of the US food and feed supply.

3.7 Hardness or vitreousness Hardness or vitreousness is an important grain quality factor for corn, affecting milling characteristics. Vitreousness is typically a subjective evaluation, based on candling, to identify the vitreous phenotypes. Kernels placed on a light box are visually scored and


assigned to arbitrary, discontinuous classes according to the ratio of vitreous to floury endosperm. Felker and Paulis (1993) proposed an image-analysis approach based on a non-destructive method for quantification of corn kernel vitreousness. Corn kernels were viewed on a light box using a monochrome video camera, and the transmitted light video images were analyzed with commercially available software. For imaging, kernels were surrounded by modeling clay to avoid light leaks around the kernels. A high degree of correlation was observed between visual scores and average grayscale values of captured video images (r 2 = 0.85). Removing the image background and correcting for kernel thickness improved the correlation (r 2 = 0.91). Erasmus and Taylor (2004) reported a rapid, non-destructive image-analysis method for determining endosperm vitreousness in corn kernels. For imaging, individual whole kernels were placed on top of round illuminated areas smaller than the projected areas of the kernels, to shine light through the kernels. A correction factor to allow constant illumination of kernels was developed to adjust kernel size variations in relation to constant light area. Significant correlations were found between corrected translucency values and endosperm yields determined by hand dissection (r = 0.79). Corrections for kernel thickness improved the correlation further (r = 0.81); however, the data spread was rather wide (r 2 = 0.65).

3.8 Seed viability Seed viability and vigor are important for the ongoing continuation of a variety. Producers would like to be assured that the corn seeds they plant will all emerge into new plants. Xie and Paulsen (2001) developed a machine vision system to detect and quantify tetrazolium staining in sectioned corn kernels for corn viability classification. The machine-vision based tetrazolium test was able to predict viability loss and therefore detrimental effects of heat on corn to be used for wet milling. Corn harvested at 20 percent and 25 percent moisture was negatively affected by drying at 70◦ C. Corn harvested at 30 percent moisture was negatively affected by heat at all drying temperatures above 25◦ C, and was much more severely affected as the drying temperature increased. Cicero and Banzatto (2003) studied the effects of mechanical damage on corn seed vigor using image analysis. Fifty seeds from three cultivars were visually selected to form a sample of whole seeds with varying degrees of mechanical damage. The seeds were X-rayed, photographed (ventral and dorsal sides), and submitted to a cold test. The cold test was used to introduce stress and hence assess the ability (vigor) of the seeds to withstand the stress. Photographs were repeated after the cold test. Images taken before and after the cold test were examined simultaneously on a computer monitor to determine the possible relationship between cause and effect. Results indicated that the method under study permits association of mechanical damage with eventual losses caused to corn seed vigor. Mondo and Cicero (2005) studied the effect of the seed position on the ears on seed quality, in terms of vigor and viability. Images obtained before and after germination were visually examined on a computer screen simultaneously to make a complete diagnosis for each seed. The results indicated that the seeds in the proximal and intermediate positions presented a similar quality and were superior to those of the distal position.


It was also reported that spherical seeds with embryonic axes presented torsions, but that neither altered nor reduced quality. However, alterations in the embryonic axes (dark, undefined stains), presented in a larger quantity in the distal region of the ear, were responsible for the loss of seed quality.

3.9 Other applications Separation of shelled corn from residues is an important task in corn harvesting and processing. Jia et al. (1991) investigated the use of machine vision for monitoring the separation of shelled corn from residues. Image analysis results showed that spectral reflectance differences in red and blue bands of the electromagnetic spectrum could be used to separate corncobs from residues. Jia (1993) proposed an automatic inspection system for grading seed maize using machine vision. Images of a number of samples of maize were acquired as the maize cobs passed through the inspection system. The samples represented the quality of inspected maize at different layers of unloading maize from a truck. Machine vision algorithms were developed to measure the amount of residues mixed with maize cobs, and the loss of kernels on cobs. Two parameters, residue mixture ratio and kernel loss ratio, were introduced as indicators of quantitative measurement of the amount of residues mixed with cobs, and kernels lost on the cobs. Seed corn is harvested and delivered on the cob with some husk still attached to avoid mechanical damage to the seeds. A husk deduction is manually estimated as the husk/corn weight ratio, for payment purposes. Precetti and Krutz (1993) developed a color machine vision system to perform real-time husk deduction measurements. They reported that a linear relationship exists between the weight ratio of the husk deduction and the surface ratio of the vision system. Variability of the machine vision system was ±1 percent compared to ±4 percent for the manual measurements.

3.10 Changing directions A number of near-infrared reflectance (NIR) spectroscopy applications have been reported in the literature for quality evaluation of corn in terms of moisture and amino acids (Fontaine et al., 2002); protein, oil and starch (Kim and Williams, 1990; Orman and Schumann, 1991); fungal infection (Pearson et al., 2006); and milling performance (Wehling et al., 1996; Dijkhuizen et al., 1998). Hyperspectral imaging appears to be a natural extension to take advantage of both the spectral and spatial information in NIR and imaging, respectively. Yu et al. (2004) used Synchrotron Fourier Transform infrared (FTIR) microspectroscopy to image the molecular chemical features of corn to explore the spatial intensity and distribution of chemical functional groups in corn tissues. Results of this study showed that FTIR images could help corn breeders in selecting superior varieties of corn for targeted food and feed markets. Cogdill et al. (2004) evaluated hyperspectral imaging as a tool to assess the quality of single maize kernels. They developed calibrations to predict moisture and oil contents in single maize kernels based on hyperspectral transmittance data in the range of 750 to 1090 nm. The moisture calibration achieved good results, with a standard error of cross-validation (SECV) of 1.2 percent and a relative performance determinant (RPD) of 2.74. The

References 419

oil calibration did not perform well (SECV = 1.38 percent, RPD = 1.45), and needs improved methods of single seed reference analysis.

4 Conclusions Corn (Zea Mays) has undergone extensive investigation with machine vision applications, and many characteristics are shown to have a high degree of detectability. Simple quality characteristics, such as size and shape, have been shown to be easily measurable using imaging techniques, while others, such as breakage and cracks, may require the additional use of dyes to reach a high degree of both detection and repeatability. However, cracked kernels arise for many reasons, and different approaches are required depending upon their origin. Imaging has the potential for mold detection and, with the exciting advancements in hyperspectral imaging, for toxin detection. In concurrence with imaging in cereals generally, the detection of corn quality factors by imaging has tended to focus on reproduction of the subjective evaluations of quality characteristics that have been used in the past to describe functionality. With the more advanced imaging technologies emerging, it can only be predicted that research will become focused on directly analyzing and describing the properties relating to the process that corn is destined for, and properties that traditional quality evaluation methods do not describe. With a growing concern for healthy food sources, there will be a need to enhance the detection of toxins to ensure safe products.

References Almeida MT, Bisby FA (1984) A simple method for establishing taxonomic characters from measurement date. Taxon, 33, 405–409. Anonymous 2001 (2000–2001) Value Enhanced Grain Quality Report. Washington, DC: US Grain Council. Baum BR, Thompson BK (1976) Classification of Canadian oat cultivars by quantifying the size-shape of their “seeds”: a step towards automatic identification. Canadian Journal of Botany, 54, 1472–1480. Brooker DB, Bakker-Arkema FW, Hall CW (1974) Drying Cereal Grains. New York: AVI. Cicero SM, Banzotto HL Jr (2003) Evaluation of image analysis in determining the relationship between mechanical damage and seed vigor in maize [in Portuguese]. Revista brasileira de semente, 25 (1), 29–36. Cogdill RP, Hurburgh CR Jr, Rippke GR (2004) Single-kernel maize analysis by nearinfrared hyperspectral imaging. Transactions of the ASAE, 47 (1), 311–320. Cuevas-Rodrigues EO, Milan-Carillo J, Mora-Escobedo R, Cardenas-Valenzuela OG, Reyes-Moreno C (2004) Quality protein maize (Zea Mays L.) temph flour through solid state fermentation process. Lebensmittel Wissenschaft und Technologie, 37, 59–67. Dicrispino K, Yao H, Hruska Z, Brabham K, Lewis D, Beach J, Brown RL, Cleveland TE (2005) Hyperspectral imagery for observing spectral signature change in Aspergillus


flavus. Proceedings of the SPIE, The International Society of Optical Engineering, 5996, 599606-1-10. Dijkhuizen A, Dudley JW, Rocheford TR, Haken AE, Eckoff SR (1998) Near-infrared reflectance correlated to 100-g wet-milling analysis in maize. Cereal Chemistry, 75 (2), 266–270. Ding K, Morey RV, Wilcke WF, Hansen DJ (1990) Corn quality evaluation with computer vision. ASAE Paper No. 90-3532, ASAE, St Joseph, MI, USA. Draper SR, Travis AJ (1984) Preliminary observations with a computer based system for analysis of the shape of seeds and vegetative structures. Journal of the National Institute of Agricultural Botany, 16, 387–395. Eckhoff SR (1992) Converting corn into food and industrial products. Illinois Research, spring/summer. Erasmus C, Taylor RN (2004) Optimising the determination of maize endosperm vitreousness by a rapid non-destructive image analysis technique. Journal of the Science of Food and Agriculture, 84, 920–930. Evers AD, Withey RP (1989) Use of image analysis to predict milling extraction rates of wheats. Food Microstructure, 8, 191–199. Felker FC, Paulis JW (1993) Quantitative estimation of corn endosperm vitreosity by video image analysis. Cereal Chemistry, 70 (6), 685–689. Floyd CD, Rooney LW, Bockholt AJ (1995) Measuring desirable and undesirable color in white and yellow food corn. Cereal Chemistry, 72 (5), 488–490. Fontaine J, Schirmer B, Horr J, (2002) Near-infrared reflectance spectroscopy (NIRS) enables the fast and accurate prediction of essential amino acid contents. 2. Results for wheat, barley, corn, triticale, wheat bran/middlings, rice bran, and sorghum. Journal of Agricultural and Food Chemistry, 50, 3902–3911. Freeman JE (1973) Quality factors affecting value of corn for wet milling. Transactions of the ASAE, 16, 671–682. Gunasekaran S, Deshpande SS, Paulsen MR, Shove DG, (1985) Size characterization of stress cracks in corn kernels. Transactions of the ASAE, 28 (5), 1668–1672. Gunasekaran S, Paulsen MR, Shove DG (1986) A laser optical method for detecting corn kernel defects. Transactions of the ASAE, 29 (1), 294–298. Gunasekaran S, Cooper T, Berlage A, Krishnan P, (1987) Image processing for stress cracks in corn kernels. Transactions of the ASAE, 30 (1), 266–271. Han YJ, Feng Y, Weller CL, (1996) Frequency domain image analysis for detecting stress cracks in corn kernels. Transactions of the ASAE, 12 (4), 487–492. Hurburgh CR (1989) The value of quality to new and existing corn uses. Agricultural Engineering Staff Papers Series FPR 89-2, ASAE-CSAE Meeting Presentation June 25–28. Jia J (1993) Seed maize quality inspection with machine vision. Proceedings of the SPIE, The International Society of Optical Engineering, 1989, 288–295. Jia J, Krutz GW, Precetti CJ (1991) Harvested corn cob quality evaluation using machine vision. ASAE Paper No. 91-6537, ASAE, St Joseph, MI, USA. Keefe PD, Draper SR (1986) The measurement of new characters for cultivar identification in wheat using machine vision. Seed Science and Technology, 14, 715–724.

References 421

Keefe PD, Draper SR (1988) An automated machine vision system for the morphometry of new cultivars and plant genebank accessions. Plant Varieties and Seeds, 1, 1–11. Kim HO, Williams PC (1990) Determination of starch and energy in feed grains by nearinfrared reflectance spectroscopy. Journal of Agricultural and Food Chemistry, 38, 682–688. Lai FS, Zayas I, Pomeranz Y (1986) Application of pattern recognition techniques in the analysis of cereal grains. Cereal Chemistry, 63 (2), 168–172. Leath MN, Hill LD (1987). Economics of production, marketing, and utilization. In Corn: Chemistry and Technology (Watson SA, Ramstad PE, eds). St Paul: American Association of Cereal Chemists, pp. 201–252. Liao K, Li Z, Reid JF, Paulsen MR, Ni B (1992a) Knowledge-based color discrimination of corn kernels. ASAE Paper No. 92-3579, ASAE, St Joseph, MI, USA. Liao K, Paulsen MR, Reid JF, Ni B, Bonificio E (1992b) Corn kernel shape identification by machine vision using a neural network classifier. ASAE Paper No. 92-7017, ASAE, St Joseph, MI, ASAE. Liao K, Paulsen MR, Reid JF, Ni B, Bonificio E (1993) Corn kernel breakage classification by machine vision using a neural network classifier. Transactions of the ASAE, 36 (6), 1949–53. Liao K, Paulsen MR, Reid JF (1994) Real-time detection of color and surface defects of maize kernels using machine vision. Journal of Agricultural Engineering Research, 59, 263–271. Litchfield JB, Shove GC (1989) Dry milling of US hard-endosperm corn in Japan. ASAE Paper No. 89-6015, ASAE, St Joseph, MI, USA. Liu J, Paulsen MR (2000). Corn whiteness measurement and classification using machine vision. Transactions of the ASAE, 43 (3), 757–763. Liu C-C, Shaw J-T, Poong K-Y, Hong M-C, Shen M-L (2005) Classifying paddy rice by morphological and color features using machine vision. Cereal Chemistry, 82, 649–653. Lookhart GL, Jones BL, Walker DE, Hall SB, Cooper DB (1983) Computer-assisted method for identifying wheat cultivars from their gliadin electrophoregrams. Cereal Chemistry, 60, 111–115. Mendoza F, Dejmek P, Aguilera JM (2006) Calibrated color measurements of agricultural foods using image analysis. Postharvest Biology and Technology, 41, 285–295. Mondo VHV, Cicero SM (2005) Using image analysis to evaluate the quality of maize seeds located in different positions on the ear [in Portuguese]. Revista brasileira de semente, 27 (1), 9–18. Neuman MR, Sapirstein HD, Shwedyk E, Bushuk W (1989a) Wheat grain color analysis by digital image processing I. Methodology. Journal of Cereal Science, 10, 175–182. Neuman MR, Sapirstein HD, Shwedyk E, Bushuk W (1989b) Wheat grain color analysis by digital image processing II. Wheat class discrimination. Journal of Cereal Science, 10, 183–188. Ni B, Paulsen MR, Reid JF (1997a) Corn kernel crown shape identification using image analysis. Transactions of the ASAE, 40 (3), 833–838. Ni B, Paulsen MR, Liar K, Reid JF (1997b) Design of an automated corn kernel inspection system for machine vision. Transactions of the ASAE, 40 (2), 491–497.


Ni B, Paulsen MR, Reid JF (1998) Size grading of corn kernels with machine vision. Applied Engineering in Agriculture, 14 (5), 567–571. Ng HF, Wilcke WF, Morey RV, Lang JP (1998a) Machine vision color calibration in assessing corn kernel damage. Transactions of the ASAE, 41 (3), 727–732. Ng HF, Wilcke WF, Morey RV, Lang JP (1998b) Machine vision evaluation of corn kernel mechanical and mold damage. Transactions of the ASAE, 41 (2), 415–420. Official Grain Grading Guide. Canadian Grain Commission, Winnipeg, Manitoba, Canada (available on-line at www.grainscanada.gc.ca). Orman BA, Schumann RA (1991) Comparison of near-infrared spectroscopy calibration methods for the prediction of protein, oil, and starch in maize grain. Journal of Agricultural and Food Chemistry, 39, 883–886. Paulsen MR, Hill LD (1985) Corn quality factors affecting dry milling performance. Journal of Agricultural Engineering Research, 31, 255. Paulsen MR, Hofing SL, Hill LD, Eckhoff SR (1996) Corn quality characteristics for Japan markets. Applied Engeering in Agriculture, 12 (6), 731–738. Pearson TC, Wicklow TG (2006) Properties of corn kernels infected by fungi. Transactions of the ASAE, 49 (4), 1235–1245. Pearson TC, Dowell FE, Armstrong PR (2006) Objective grading and end-use property assessment of single kernels and bulk grain samples. 2005 Annual Progress Reports (NC-213): Management of Grain Quality and Security in World Markets, pp. 67–71 (available on-line at http://www.oardc.ohio-state.edu/nc213/PR05.pdf). Pierce RO, Salter KL, Jones D (1991) On-farm broken corn levels. Applied Engineering in Agriculture, 7 (6), 741–745. Precetti CJ, Krutz GW (1993) A new seed corn husk deduction system using color machine vision. ASAE Paper No. 93-1012, ASAE, St Joseph, MI, USA. Rausch KD, Eckhoff SR, Paulsen MR (1997) Evaluation of the displacement value as a method to detect reduced corn wet milling quality. Cereal Chemistry, 74 (3), 274–280. Reid JF, Kim C, Paulsen M (1991) A computer vision sensor for automatic detection of stress cracks in corn kernels. Transactions of the ASAE, 34 (5), 2236–2244. Sapirstein HD, Bushuk W (1985a) Computer-aided analysis of gliadin electrophoregrams. I. Improvement of precision of relative mobility determination by using a three reference band standardization. Cereal Chemistry, 62, 373–377. Sapirstein HD, Bushuk W (1985b) Computer-aided analysis of gliadin electrophoregrams. III. Characterization of the heterogeneity in gliadin composition for a population of 98 common wheats. Cereal Chemistry, 62, 392–398. Sapirstein HD, Bushuk W (1989) Quantitative determination of foreign material and vitreosity in wheat by digital image analysis. Proceedings ICC Symposium: Wheat End-Use Properties: Wheat and Flour Characterization for Specific End-Uses, June 13–15, Lahti Finland. Helsinki: University of Helsinki, Department of Food Chemistry and Technology. Sapirstein HD, Kohler JM (1999) Effects of sampling and wheat grade on precision and accuracy of kernel features determined by digital image analysis. Cereal Chemistry, 76, 110–115. Shahin MA, Symons SJ (2001) A machine vision system for grading lentils. Canadian Biosystems Engineering, 43, 7.7–7.14.

References 423

Shahin MA, Symons SJ (2005) Seed Sizing from images of non-singulated grain samples. Canadian Biosystems Engineering, 47, 3.49–3.55. Shatadal P, Jayas DS, Bulley NR (1995a) Digital image analysis for software separation and classification of touching grains: I. Disconnect algorithm. Transactions of the ASAE, 38, 635–643. Shatadal P, Jayas DS, Bulley NR (1995b) Digital image analysis for software separation and classification of touching grains: II. Classification. Transactions of the ASAE, 38, 645–649. Shotwell OL, Hesseltine CW (1981) Use of bright greenish yellow fluorescence as a presumptive test for aflatoxin in corn. Cereal Chemistry, 58 (2), 124–127. Steenhoek LW, Precetti CJ (2000) Vision sizing of seed corn. ASAE Paper No. 00-3095, ASAE, St Joseph, MI, USA. Steenhoek LW, Misra MK, Batchelor WD, Davidson JL (2001) Probabilistic neural networks for segmentation of corn kernel images. Applied Engineering in Agriculture, 17 (2), 225–234. Symons SJ, Dexter JE (1991) Computer analysis of fluorescence for the measurement of flour refinement as determined by flour ash content, flour grade color, and tristimulus color measurements. Cereal Chemistry, 68, 454–460. Symons SJ, Dexter JE (1992) Estimation of milling efficiency: prediction of flour refinement by the measurement of pericarp fluorescence. Cereal Chemistry, 69, 137–141. Symons SJ, Dexter JE (1993) Relationship of flour aleurone fluorescence to flour refinement for some Canadian hard common wheat classes. Cereal Chemistry, 70, 90–95. Symons SJ, Dexter JE (1996) Aleurone and pericarp fluorescence as estimators of mill stream refinement for various Canadian wheat classes. Journal of Cereal Science, 23, 73–83. Symons SJ, Fulcher RG (1987) The morphological characterization of seeds using digital image analysis. Proceedings of the 37th Australian Cereal Chemistry Conference, pp. 54–57. Symons SJ, Fulcher RG (1988a) Determination of wheat kernel morphological variation by digital image analysis: II. Variation in cultivars of soft white winter wheats. Journal of Cereal Science, 8, 219–229. Symons SJ, Fulcher RG (1988b) Relationship between oat kernel weight and milling yield. Journal of Cereal Science, 7, 215–217. Symons SJ, Fulcher RG (1988c) Determination of variation in oat kernel morphology by digital image analysis. Journal of Cereal Science, 7, 219–228. Symons SJ, Dexter JE, Matsuo RR, Marchylo BA (1996) Semolina speck counting using an automated imaging system. Cereal Chemistry, 73, 561–566. Symons SJ, Van Schepdael L, Dexter JE (2003) Measurement of hard vitreous kernels in durum wheat by machine vision. Cereal Chemistry, 80, 511–517. Thompson RA, Foster GH (1963) Stress cracks and breakage in artificially dried corn. Marketing Research Rep. No. 631, October. Washington, DC: USDA, AMS, TFRD. Travis AJ, Draper SR (1985) A computer-based system for the recognition of seed shape. Seed Science Technology, 13, 813–820.


Watson SA (1987) Structure and composition. In Corn Chemistry and Technology (Watson SA, Ramstad, PE, eds). St Paul: American Association of Cereal Chemists, pp. 53–78. Watson SA, Hirata Y (1962) Some wet-milling properties of artificially dried corn. Cereal Chemistry, 39, 35–44. Wehling RL, Jackson DS, Hamaker BR (1996) Prediction of corn dry-milling quality by near-infrared spectroscopy. Cereal Chemistry, 73 (5), 543–546. Winter P, Wood H, Young W, Sokhansanj S (1997) Neural networks and machine vision team up to grade corn. Vision Systems Design, October, 28–33. Xie W, Paulsen MR (2001) Machine vision detection of tetrazolium staining in corn. Transactions of the ASAE, 44 (2), 421–428. Xie F, Pearson T, Dowell FE, Zhang N (2004) Detecting vitreous wheat kernels using reflectance and transmittance image analysis. Cereal Chemistry, 81, 594–597. Yu P, McKinnon JJ, Christensen CR, Christensen DA (2004) Imaging molecular chemistry of pioneer corn. Journal of Agricultural and Food Chemistry, 52, 7345–7352. Zayas I., Pomeranz Y, Lai FS (1985) Discrimination between Arthur and Arkan wheats by image analysis. Cereal Chemistry, 62, 478–480. Zayas I., Lai FF, Pomeranz Y (1986) Discrimination between wheat classes and varieties by image analysis. Cereal Chemistry, 63, 52–56. Zayas I, Converse H, Steele JL (1990) Discrimination of whole from broken corn kernels with image analysis. Transactions of the ASAE, 33 (5), 1642–1646. Zayas I, Bechtel DB, Wilson JD, Dempster RE (1994) Distinguishing selected hard and soft red wheats by image analysis of starch granules. Cereal Chemistry, 71, 82–86.

Quality Evaluation of Pizzas Cheng-Jin Du and Da-Wen Sun Food Refrigeration and Computerised Food Technology, University College Dublin, National University of Ireland, Dublin 2, Ireland

1 Introduction With pizza being one of the more popular consumer foods, pizza markets in Europe, America, and other continents have been boosted by the trend towards international cuisine and convenience foods (Anonymous, 1994). As a result, pizza production has been increasing at unprecedented momentum, and is expected to increase further in the next decade in response to a growing world population. For example, the frozen pizza market increased by almost 24 percent between 1999 and 2002, to a83 million, according to figures from the Irish food board. Compared with the traditional homemade method of production, the modern method manufactures pizzas automatically and production efficiency is thus greatly increased. In today’s highly competitive market, quality is a key factor for the modern pizza industry because the high quality of products is the basis for success. A challenging problem faced by the manufacturers is how to keep producing consistent products under variable conditions, especially with the inherent sensitivity of pizza-making. Manual evaluation methods are tedious, laborious, costly, and time-consuming, and are easily influenced by physiological factors, thus inducing subjective and inconsistent evaluation results. For example, the method used by the Green Isle Foods Company, of Naas in Ireland (a leading pizza-maker in Ireland that had a 58 percent market share in frozen pizza in 1996), for pizza base evaluation is assessment by a human inspector, who compares each base with a standard one Given the huge number of bases that move along the production line at an appreciable speed, it is hard to believe that such a standard can be maintained purely by visual inspection by a number of personnel over a period of several hours. To satisfy the increased awareness, sophistication, and expectations of consumers, and to achieve success in a growing and competitive market, it is necessary to improve the methods for quality evaluation of pizza products. If quality evaluation is achieved automatically using computer vision, the production speed and efficiency Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


18

428 Quality Evaluation of Pizzas

can be improved, as well as evaluation accuracy, with an accompanying reduction in production costs (Sun and Brosnan, 2003a). According to the pizza expert at Green Isle Foods, manufacturing of pizzas can generally be broken down into three main steps – pizza base production, sauce spread, and topping application. The basic recipe for the dough used in pizza bases consists of flour, water, dry yeast, salt, oil, and sodium stearoyl-2-lactylate (Matz, 1989). First, each ingredient is weighed and they are then mixed together. After the dough has been allowed to rise, dough units are scaled and rounded before being flattened and rolled. Finally, sauces are spread on the base and toppings are applied to form the final product. In this chapter, the application of computer vision for pizza quality evaluation will be discussed according to these manufacturing stages.

2 Pizza base production In some literature the pizza base is also called the pizza crust. It comprises 55 percent of the weight of pizza (Lehmann and Dubois, 1980). Although the crust might not seem very exciting, it forms the basis upon which all the other parts come together (Burg, 1998). Furthermore, pizza products are normally categorized according to the production methods of the crust – for example, if the base is prepared by the leavening method, the pizza can be classified as “yeast-leavened” or “chemically-leavened” (i.e. with soda added). Therefore, it seems that the pizza base has attracted more attention in the literature than have the pizza sauce and the topping. There are two basic procedures for pizza base production: either the dough is divided, rounded, and pressed into discs, or it is rolled out in a continuous sheet from which circles are cut. The latter method produces uniform circles (Matz, 1989). In contrast, the former method can give a better texture, but at the cost of the fixed size and perfectly round shape, which it is very hard to obtain naturally. However, precise size and shape are important to consumers; this is very clear when dozens of pizzas are on the same shelf and the customers can choose their own. Therefore, the visual quality of the pizza base, including its size and shape, is one of the main aspects considered by the pizza industry. Currently, most pizza producers have set up rigorous standards for the size and shape of pizza bases. For example, Green Isle Foods specifies that its 9-inch bases have dimensions of 223 × 223 mm, and any deviation from these dimensions is unacceptable. Furthermore, after heat processing a large surface area of the pizza shell becomes brown, and this brownness correlates with the nutritive value of the pizza, such as its lysine content (Unklesbay et al., 1983). To evaluate the size, shape, and color features of pizza bases automatically, computer vision techniques could be applied.

2.1 Feature extraction 2.1.1 Size The measurements of an object most commonly used in food quality evaluation are the area, perimeter, length and width. The most basic convenient measurement for the size of pizza bases is the area. For a pixel-based representation of the pizza base image,

Pizza base production 429

this is the number of pixels within the area. To distinguish the pizza base area from the background, a thresholding-based image segmentation technique can be employed. Thresholding-based segmentation is a particularly effective technique in the scenario where solid objects are laid upon a contrasting background, as in pizzas. Moreover, it is computationally simple and is capable of different regions that have closed and connected boundaries. The optimal value of the threshold can be obtained from the gray-level histogram of the image. With the optimal threshold, all pixels at or above the threshold level are assigned to the pizza base, while all pixels below the threshold fall outside the pizza base and are set as background. However, in some cases use of the global thresholding technique alone is not enough to segment the pizza base from the background because the contrast varies within the image. The automatic thresholding technique developed by Panigrahi et al. (1995) is an alternative method that can be applied to segment the background from the images of the pizza base. Based on the segmented image, the area of the pizza base can be straightforwardly determined by counting the number of pixels assigned to the pizza base. Another way to obtain the area of pizza is first to determine the Feret diameters (the distance between two tangents on opposite side of the area perpendicular to a given direction) at 1◦ intervals, to a maximum of 180◦ . The area of the pizza can then be calculated using the average Feret diameter. The advantage of this method is that it can obtain the areas of the smallest and largest rounds of the pizza for further shape analysis, as shown in the work of Sun and Brosnan (2003a). 2.1.2 Shape

In practice, producing a perfectly round pizza base is not easy because current technology cannot guarantee the visual quality of each individual pizza within a batch in large-scale manufacture. Generally, there are three types of pizza base defects: flowing base, poor alignment, and poor pressing. A flowing base is simply where one side of the dough has spread out (flowed) more on the tray, and the pizza is therefore longer in one dimension rather than having a consistent diameter. Poor alignment occurs if the dough-ball is not centered when being pressed out, and hence the pizza does not have a round shape (it would generally have a straight side, or just not be completely round). Poor pressing is characterized by an out-of-shape base, which is not completely round; however, such pizzas are usually considered acceptable. Figure 18.1 illustrates the three defects, along with a standard base. Three steps of an image processing algorithm can be developed to extract pizza base contours from digital images – noise reduction by a median filter, thresholdingbased segmentation, and edge detection. In order to improve the quality of the captured image, operations need to be performed to remove or decrease degradations suffered in its acquisition. A median filter is a non-linear filtering technique which allows the edges to be preserved while filtering out the unwanted noise, and is suitable for removing possible noise within the pizza base image. A thresholding-based image segmentation method can then be applied to separate the pizza base from the background. The segmentation step is necessary to obtain a closed and continuous boundary of the pizza base, which it is difficult to obtain directly by traditional edge-detection technology. Thus, the boundary becomes a set of interior points, each of which has at least one


(a)

(b)

(c)

(d)

Figure 18.1 Illustrated images of pizza base: (a) flowing base; (b) poor alignment; (c) poor pressing; (d) standard.

neighbor outside the pizza base. From the segmented image, the shape of the pizza base can be detected by the Canny edge detector (Canny, 1986) with selected low and high hysteresis thresholds. From the extracted shape and the segmented image of the pizza base, several shape features can be calculated to characterize it, such as area ratio, aspect ratio, eccentricity, roundness, and circularity, which are defined by the following equations: Area ratio =

Area Max diameter · Min diameter

Aspect ratio =

Max diameter Min diameter

Eccentricity = Roundness =

(18.1)

(18.2)

SemiMinor 2 SemiMajor 2

(18.3)

4 · Area π · Max diameter 2

(18.4)

1−

Pizza base production 431

Perimeter 2 (18.5) Area Considering that the circularity index alone is not enough for adequate shape analysis, two other criteria were introduced (Sun and Brosnan, 2003a): Circularity =

• •

Spatial ratio I (SRI) = pizza base area/area of the smallest round which can hold the pizza base Spatial ratio II (SRII) = area of the biggest round which can be fitted into by the pizza base/pizza base area.

Besides the above shape features, which involve a combination of size measurements, a shape description technique that is independent of size measurement, such as the Fourier descriptor, can be applied to describe the boundary of the pizza base. The Fourier transform of one cycle of the boundary function is an alternative representation of the associated object’s shape, which can characterize the magnitude of changes in the shape frequency in the spatial domain. Using Fourier transform, the boundary function spectrum can be low-pass (only allowing low-frequency impulses to pass) filtered without destroying the characteristic shape of the object. Only the amplitudes and phases of the low-frequency impulses in the spectrum, i.e. the low-order Fourier coefficients, are required to characterize the basic shape of the object. For example, Du and Sun (2004) found that seven coefficients of the Fourier transform contained most of the information regarding the shape of pizza base, and were adequate for representing its characteristics. These values are candidates for shape descriptors. 2.1.3 Color

Color features of an object can be extracted by examining every pixel within the pizza base boundaries. For pizza base production, the determination of the color information regarding the total surface area after heat processing, especially the brownness level, is important, because excessive heating of proteins produces charring and results in irreversible lysine loss, caused by deamination or decarboxylation (Anglemier and Montgomery, 1976). As an objective measurement method, a computer vision approach has been developed by Unklesbay et al. (1983) for determining the relative degree of browning of pizza shells, prepared with and without soy flour. The percentage of total area of each brown intensity level is obtained by constructing a histogram of the gray level representing the image of the pizza base. In their study, the usefulness of color information in the prediction of available lysine content is also confirmed. It is shown that the developed technique is very promising in cases where a rapid, non-destructive test for available lysine in baked dough is needed.

2.2 Classification In practice, classification of pizza bases into acceptable and unacceptable levels can satisfy the general requirements of industrial application. Sun and Brosnan (2003a) classified pizza bases according to the results of the four indices, i.e. area, SRI, SRII, and circularity. In this method, evaluation of the standard samples is first performed to allow comparison. From the results, the lowest values of the area, SRI and SRII,


and the poorest circularity in the analyzed samples, are considered as the basis for classification into acceptable and unacceptable quality levels. Any pizza base with results lower than the corresponding limit area, SRI and SRII, or larger circularity than the limit, are considered to be defective. After studying the results, based on the limits, an inaccuracy of 13 percent was obtained when a combination of these indexes was used (Sun and Brosnan, 2003a). In practice, as high-speed processing is important, the order of the four indices used when running the program has to be determined to produce an efficient and effective quality analysis system. The most effective classification is attained by use of the circularity index. However, even though the SRI and SRII are much less efficient, they should still be used for the classification of some samples. Hence the best order in which to apply the indexes is circularity, area, SRII and SRI, to determine acceptable and defective pizza bases. Another promising method of classifying pizza bases as acceptable or unacceptable is based on the Fourier transform technique. Using the Fourier coefficients of each image as inputs, a variety of classifiers can be used for classification. Three parametric classifiers and four non-parametric classifiers were compared with support vector machine (SVMs) classifiers for evaluating their classification performance (Du and Sun, 2004). The three parametric classifiers are: 1. The least squares classifier (Mendel and Fu, 1970), where the classification criterion is the minimum Euclidean distance between the unknown entry and the mean values of each of the other classes 2. The maximum likelihood classifier (Schowengerdt, 1983), where the maximum likelihood equation uses the Mahalanobis minimum distance 3. The regularized discriminant analysis classifier (Friedman, 1989), which employs the quadratic discriminant function with regularized covariance matrices. The four non-parametric classifiers are: 1. The K nearest-neighbor classifier (Therrien, 1989), where the classification rule is the minimum Euclidean distance between the unseen sample and the mean values of the other class 2. The localized boosting classifier (Meir et al., 2000), which is an incremental greedy learning algorithm based on a maximum-likelihood approach and the mixture of expert architecture 3. The C4.5 classifier (Quinlan, 1993), which is a decision tree/rule algorithm 4. The radial basis function network classifier (Mark, 1996), which is a type of artificial neural network associated with radial basis functions (RBF). The performance of the classifiers is evaluated by using different parameter configurations. The parameters of each algorithm with the best performance in separating the pizza bases are selected, and the corresponding classification accuracy results are presented. The results show that the non-parametric classifiers perform better than the parametric classifiers; of the latter, the least squares classifier is the best method, with a classification rate of 90 percent. Among the non-parametric classifiers, the polynomial SVM and RBF SVM classifiers perform better than all other classifiers, with

Pizza sauce spread 433

classification accuracies of 95.0 percent and 98.3 percent respectively. In SVMs, the best overall classification accuracy result achieved is 98.3 percent using the RBF SVM classifier. In terms of the classification time, the non-parametric classifiers are more expensive than parametric classifiers. The SVM classifiers are roughly comparable to C4.5 and RBF network classifiers. For multi-classification of pizza base samples into four shape-quality levels (flowing base, poor alignment, poor pressing, and standard), several SVM algorithms have recently been applied. Supposing that each sample is represented by shape features such as area ratio, aspect ratio, eccentricity, roundness, and Fourier descriptors, one approach is to construct and combine several binary SVM classifiers, such as oneversus-all (Vapnik, 1998), one-versus-one (Kreßel, 1999), and directed acyclic graph (DAG) SVM (Platt et al., 2000). Another approach is to use a single optimization formulation (Crammer and Singer, 2001). Owing to its computational expensiveness and complexity, single SVM formulation is usually avoided. According to the results obtained by Du and Sun (2005a), the one-versus-all method performs worst among the three multi-classification methods. The DAG method takes less time than the one-versus-one method, while its classification accuracy is worse than the latter. Therefore, the one-versus-one and the DAG method have similar performance for multi-classification of pizza bases.

3 Pizza sauce spread The sauce is critical, and can be a signature part of the pizza (Burg, 1998). Therefore, the quality of pizza sauce spread is an influential factor when evaluating the whole quality of a pizza. A strict inspection of the visual quality of the sauce coating on pizzas is significant for pizza manufacturers. Flowing pizza sauce means that poor spreading happens often in pizza factories. A pizza with poorly-sauce might have an adverse affect on consumers, arising from concerns about the sanitary conditions of manufacture and the mouth-feel quality of the pizza, etc. Besides the visual quality, sauce spread also affects other attributes of the pizza. The influence of pizza sauce on the pH and the moisture, salt and calcium contents during refrigerated storage of pizzas was studied by combining cheese shreds with the pizza sauce for direct contact in a model system (Wang et al., 1998). There are five levels of sauce quantities laid on pizza: even spread, reject overwipe, acceptable overwipe, reject underwipe, and acceptable underwipe (see Figure 18.2). Compared with the visual inspection of pizza bases, inspection of the pizza sauce spread is much more complex. The quality-control personnel in Green Isle Foods confirmed that there is no feasible method of judging the quality of sauce coating accurately; instead, an informal standard involving the consideration of the area of sauce on pizzas is used. In fact, their grading results for the samples indicate that the inspection is based on human experience. From the sensory point of view, the sauce thickness, distribution and color, and lighting conditions, might all influence a consumer’s feelings regarding the quantity of sauce on a pizza. Because pizza sauce has a paste-like consistency, when it is applied to pizzas it shows only very slight three-dimensional characteristics when


(a)

(b)

(c)

(d)

(e) Figure 18.2 Illustrated images of pizza sauce spread: (a) reject underwipe; (b) acceptable underwipe; (c) even spread; (d) acceptable overwipe; (e) reject overwipe.

observed from above or the side. This characteristic of pizza sauce applied to pizzas causes particular difficulty in the task of inspection. As one of the most promising methods for objective assessment, computer vision has been successfully applied for evaluation of the appearance quality of pizza sauce spread.

3.1 Color feature extraction Color is invariant with respect to camera position and pizza orientation, and has proven successful for quality evaluation of pizza sauce spread. The images of pizza


sauce spread are normally saved in the three-dimensional RGB (red, green, and blue) color space. Unfortunately, the RGB color space used in computer graphics is devicedependent, i.e. it is designed for specific devices, such as a cathode-ray tube (CRT) display. Therefore, the RGB space has no accurate definition for a human observer, where the proximity of colors in the space does not indicate color similarity in perception. Compared to RGB color space, HSV (hue, saturation, and value) is an intuitive color space. It is a user-oriented color system based on the artist’s idea of tint, shade, and tone. For efficient visual appraisal of pizza sauce spread, the RGB color space is transformed to HSV space. To characterize the color features of pizza sauce spread, Sun and Brosnan (2003a) analyzed images of pizza sauce spread based on a simple thresholding segmentation method. By setting the values of H, S, and V color components in the ranges [220, 14], [0, 125], and [0, 200], respectively, segmentation of pizza sauce from the pizza base is achieved. Then segmentation of the heavy/light zones of pizza sauce is accomplished by setting the HSV values to [2, 14], [53, 125], and [106, 200], respectively. After that, two indices are chosen for evaluating the quantity of pizza sauce spread on pizza bases: Sauce area percentage (%) (SAP) = (sauce area/pizza base area) × 100% Heavy area percentage (%) (HAP) = (heavy zone area/sauce area) × 100% The greatest disadvantage of the above method is that it is likely to become tuned to one type of image (e.g. a specific sensor, scene setting, illumination, and so on), which limits its applicability. The performance of the algorithm degrades significantly when the color and the intensity of the illuminant are changed. To overcome this disadvantage, a hybrid method was developed by Du and Sun (2005b). First, to reduce the effect of illumination on the system, the value component (V) is ignored for colorfeatures extraction from pizza sauce spread. Then, a vector quantifier (Gray, 1984) is designed to quantify the remaining two-dimensional space, i.e. hue and saturation, and yields a collection of 256 distinct colors. After that, a color histogram is employed to represent the distribution of color features in the image of the pizza sauce spread. In real implementation, the quantified 256-dimensional vectors are still too numerous to allow fast and accurate classification. Meanwhile, there are a number of portions of the quantified color histogram with zero value. Principal component analysis (PCA) is applied to reduce the dimensionality of the quantified vectors. The first few principal components are used to represent the color information of pizza sauce spread. In addition to the color features represented with the first few principal components, the mean and standard deviation of the H (hue) and S (saturation) color components can also be computed. The means characterize the average color properties of pizza sauce spread and topping, while the standard deviations provide a measurement of color variation.

3.2 Classification Based on the color features extracted, the images of pizza sauce spread are quantitatively characterized. Both fuzzy logic and SVM have been demonstrated to be feasible for classification of pizza sauce spread (Sun and Brosnan, 2003a; Du and Sun, 2005b).


3.2.1 Fuzzy logic

Fuzzy logic embodies the nature of a human mind in some sense. To illustrate the different parts of fuzzy logic, the quantity of pizza sauce on a pizza base is chosen as the aspect over which the fuzzy sets are defined. The quantity of pizza sauce on a pizza is fuzzy space X – namely, the fuzzy universe. Pizzas with the five different levels of quantity of pizza sauce can be regarded as five fuzzy sets. However, there is no universal method for fuzzy-set establishment. In the research of Sun and Brosnan (2003a), the five different levels of sauce spread are defined in linguistic terms by the quality personnel. The terms themselves contain information about the corresponding construction of the fuzzy sets. For instance, acceptable overwipe and reject overwipe are both in the range of overwipe, although at different intensity levels. Therefore, the two levels should be interrelated when converted into fuzzy sets. This overlapping may also occur for other classifications and/or sets. For the creation of the fuzzy sets, several membership functions can be used – such as triangular function. The fuzzy-set development can be described as follows. First, it is necessary to define the scale of each fuzzy set. In the SAP fuzzy universe, for example, there are five SAP values in each set. The minimum value of one set is used as the left boundary and the maximum as the right boundary of that set. The next step is to locate the point with full membership for a triangular membership function. In this case, the mean of the one set (five values) is chosen as the point with a membership value equal to 1. Finally, the fuzzy set is constructed by setting the left and right boundaries as zero membership and drawing the triangle by connecting the known three points. The fuzzy sets built in this way have characteristics such that when the intensity of an evaluation index (value of a fuzzy variable) moves from left to right in the universe, its membership grade decreases in one set and increases in another in a linear fashion, with the membership grades in the two adjacent and partly overlapping sets having a sum of 1. In the fuzzy universes of SAP and HAP, the fuzzy-set construction is different because the SAP and HAP are not related to each other. It is considered that the overlapping condition of a fuzzy universe can reflect the fuzzy degree of a universe. Therefore, the ambiguous degree (AD), which is a measurement of the fuzzy degree, is based on the overlapping area and defined as follows: Ambiguous degree (AD) = total overlapping area/total area of the five fuzzy sets For SAP and HAP, two AD values can be obtained, i.e. AD1 and AD2, respectively. The greater the AD value, the less of the corresponding index is taken into account for judging pizza sauce quantity in human assessment. The results reported by Sun and Brosnan (2003a) show that the value of AD1 is less than that of AD2, which indicates that SAP is more efficient than HAP as a fuzzy index. Therefore, fuzzy evaluation score (FES) can be developed to reflect the sauce spread quantity, using fuzzy logic, which is defined as FES =

n i=1

[x n /(ADn × MF n )]

(18.6)


where n denotes the evaluation indices; xn is the fuzzy variable value for index n; the magnitude factor (MF) is the mean of the total variable values for index n. Through FES calculation, each pizza sample can be given a final score of sauce quantity based on fuzzy logic, which can subsequently be converted into a ranking of pizza sauce spread quality. In an ideal situation, the applied classification of ranking should show an increase in rank of an individual set in a random manner. However, the reported results obtained from the fuzzy logic classification do not agree with the ideal case (Sun and Brosnan, 2003a). The misclassification rate is 8 percent when only two levels of quality are considered, i.e. acceptable quality and defective quality. However, the misclassification rate increases to 52 percent when the samples are classified into five quality levels. These misclassifications may be accounted for by algorithm inaccuracy. Algorithm inaccuracies originate from the use of only two indexes for the calculation of the FES, with the possibility that further elements of the pizza sauce quality may influence consumer judgment. 3.2.2 Support vector machine

Classification of pizza sauce spread into acceptable and unacceptable quality levels by SVM can be described as the task of finding a classification decision function. The SVM fixes the decision function based on structural risk minimization instead of the minimization of the misclassification on the training set to avoid the overfitting problem. It performs binary classification by finding maximal margin hyperplanes in terms of a subset of the input data between different classes. The subset of vectors defining the hyperplanes is called support vectors. If the input data are not linearly separable, the SVM first maps the data into a high (possibly infinite) dimensional feature space by using the kernel trick (Boser et al., 1992). Besides a linear kernel, polynomial kernels and Gaussian radial basis function (RBF) kernels are usually applied in practice. It then classifies the data by the maximal margin hyperplanes. Furthermore, the SVM is capable of classification in high-dimensional feature space with fewer training data. The results presented in Du and Sun (2005b) have demonstrated the ability of the SVM to classify pizza sauce spread into acceptable and unacceptable quality levels. With the extracted color features as the input, the best classification accuracy of 96.67 percent is achieved by the polynomial SVM classifiers, and 95.00 percent accuracy is obtained using the RBF SVM classifier. Multi-classification of pizza sauce spread is not an easy problem, and this can be attributed to the overlapping in some classes (Sun and Brosnan, 2003a). Three methods which combine several binary SVM classifiers are employed to perform the multiclassification task, including one-versus-all, one-versus-one, and DAG SVMs (Du and Sun, 2005a). The best classification accuracy is 87.5 percent for both the DAG and the one-versus-one methods. The performances of the one-versus-one method and DAG method are very similar, with the same optimal parameters and classification accuracy. The only differences are the number of support vectors and the time taken; the oneversus-one method returns slightly more support vectors and is a little bit slower than the DAG method. The performance of the one-versus-all method is the worst among the three methods for multi-classification of pizza sauce spread.


4 Application of pizza toppings The most attractive aspect of a pizza is its toppings. Cheese shreds, meats, and vegetables are traditional toppings. Originally, only low-moisture Mozzarella cheese, semi-skimmed or full-fat, was used for pizza making. With the development of pizza products, a variety of other cheeses have become commonly used in pizza making, including Cheddar and processed cheeses. The most popular meat items are sausage and pepperoni, while other suitable meats include ham, bacon, chicken, Canadian bacon, shrimp, and other seafood. There is a vast array of vegetables and herbs used for pizza toppings, among which mushrooms enjoy a high popularity (Burg, 1998). Additionally, some ingredients that were not used previously have now been introduced into pizza products to tempt people with differing tastes. From the viewpoint of visual quality, topping is regarded as having a connection with local culture and as a sign of imagination by some pizza producers (Burg, 1998; Matz, 1989). The quantity and the distribution of the toppings on a pizza will have a great effect on the customers’ perception of quality and likelihood to purchase. Moreover, toppings are a value-added part of the offering that reflect flavor, trends, profitability, and nutritional aspects. Regarding the shredded cheese topping, a study was conducted by Guinee and O’Callaghan (1997) to measure the stretchability of cheese on cooked pizza base. The stretchability of molten low-moisture Mozzarella cheese was found to be greater than that of Cheddar cheese. Proctor and Heidebrecht (1998) invented a multilayered pizza product, where the moisture content of the cheese was controlled in order to reduce the amount of steam produced during baking. Recently, the melting, browning, and oiling properties of pizza cheese were investigated by Wang and Sun (2002, 2003, 2004a, 2004b), using computer vision. The general visual features of an acceptable pizza should include a regular overall color histogram, uniform color of each individual topping, a predefined area percentage for each topping, and an even distribution of individual toppings (Sun, 2000). Normally, pizza-topping samples can be graded into three acceptable levels (acceptable underwipe, even spread, and acceptable overwipe) and two unacceptable levels (reject underwipe and reject overwipe). Figure 18.3 shows five illustrated pizza images topped with shredded cheese, ham, and mushroom, one for each quality level. Inspection of an actual pizza-topping image by a computer vision system is a very difficult task because a pizza has many different toppings; moreover, each individual topping has non-uniform color and shape, some different toppings (such as bacon, red pepper, and tomato sauce) are of similar color, and overlap of toppings occurs in every pizza. Using computer vision, the visual quality of pizza toppings can be evaluated and classified according to their color information, topping percentage, and distribution.

4.1 Evaluating color Color is an influential attribute and powerful descriptor that affects the quality of pizza toppings. Color vision offers a tremendous amount of spatial resolution that can be used to quantify the color distribution of different ingredients, and has proven successful for objective evaluation of the qualities of many types of food products, including pizza.

Application of pizza toppings 439

(a)

(b)

(c)

(d)

(e) Figure 18.3 Illustrated images of pizza topping: (a) reject underwipe; (b) acceptable underwipe; (c) even spread; (d) acceptable overwipe; (e) reject overwipe.


4.1.1 Color feature extraction

Being a relatively inexpensive method of image acquisition, charge-coupled device (CCD) cameras are frequently employed in computer vision systems for pizza quality evaluation. As mentioned previously, the image acquired is generally in the threedimensional RGB color space, which is not perceptually uniform, and the proximity of colors does not indicate color similarity. Research shows that color space transformation is a powerful tool for color features extraction of pizza topping (Du and Sun, 2005c). To study the effect of choosing appropriate color space transformations on the classification performance of pizza toppings, five different transformations of RGB color space were evaluated – normalized RGB (NRGB), HSV (hue, saturation, and value), I1I2I3 (Ohta et al., 1980), L*a*b*, and YCbCr (Mathworks, 1998). Of these color spaces, HSV color space is intuitive, i.e. its components can be easily related to the physical world. The others are unintuitive; their components have only an abstract relationship with the perceived color, and therefore they have no accuracy of definition for human observers. The results obtained by Du and Sun (2005c) show that the classification rates using NRGB color space transformation are significantly lower than those of the other color space transformations with all the classification methods, which means that the pizzatopping samples cannot be characterized efficiently by the NRGB color space.

4.1.2 Classification At present, there are a number of approaches that have been or are being developed for the classification of food products. These approaches include classical methods such as C4.5 (Quinlan, 1993) and artificial neural networks, and emerging methods such as SVM. In the work of Du and Sun (2005c), the performance of SVM classifiers on the binary classification of pizza toppings using different color space transformations is compared with two other classical classification approaches, i.e. the C4.5 classifier and the RBF_NN (radial basis function neural network) classifier (Bishop, 1995). It is found that pizza-topping samples cannot be simply separated by the linear SVM classifier. The RBF_NN classifier with six hidden units, the polynomial SVM classifier [2, 3], and the RBF SVM classifier with σ = 2.0 can achieve a better classification rate using HSV, I1I2I3, L*a*b*, and YCbCr color space transformation, while the C4.5 classifier with p = 1 can obtain better classification accuracy using I1I2I3 color space transformation. Higher classification accuracies, of 93.3, 86.7, 96.7, and 90.0 percent, can be obtained by using the C4.5 classifier, the RBF_NN classifier, the polynomial SVM classifier, and the RBF SVM classifier, respectively. The results indicate that the polynomial SVM classifier with proper color space transformation selection outperforms the C4.5 classifier and the RBF_NN classifier for the classification of pizza toppings. Using the selected color features as input, the three SVM methods as detailed in section 2.2 are applied to multi-classify pizza topping (Du and Sun, 2005a). Similar classification accuracies of 79.17 percent, 80.00 percent, and 80.83 percent are obtained by the one-versus-all, DAG, and one-versus-one methods, respectively. The one-versus-all method returns twice as many support vectors and takes twice as long


as the other two methods. Besides having the best classification accuracy, the oneversus-one method performs best in returning less support vectors and taking less time. Thus, the one-versus-one method is more suitable for multi-classification of pizza topping.

4.2 Evaluating topping percentage and distribution The topping percentage and distribution are the key features of pizza quality inspected by pizza manufacturers. The extraction of these features depends on the segmentation quality of pizza-topping images. Since the non-uniform color and irregular shapes of toppings make it hard to control light reflection during the image-capturing process, segmentation of pizza-topping images is difficult. Based on the segmented images, the evenness of topping distribution can be evaluated. 4.2.1 Pizza-topping segmentation

Image segmentation separates a pizza-topping image into its constituent objects. This is a challenging task because of the complex visual features and varieties of toppings. Traditional segmentation methods, such as thresholding-based, gradient-based, and region-based approaches, have been found to be only partly suitable for most pizza images. Thresholding-based methods provide a simple and fast technique for distinguishing a certain part of an image from the remainder with a gray-scale level or color intensity lower than a certain value. For pizza-topping images, the RGB intensity ranges of different toppings generally overlap – for example, the red intensity ranges may be 157–229, 151–188, and 148–205 for pepperoni, tomato sauce, and red pepper, respectively. Therefore, thresholding techniques are only suitable for very simple pizza-topping images. The gradient-based approach attempts to find the edges directly by their high gradient magnitudes. The application of gradient-based segmentation is also limited because completed boundaries are difficult (and sometimes impossible) to trace or follow in most pizza images. On the contrary, region-based segmentation is a more general-purpose method, performed by grouping pixels or sub-regions into larger regions according to a set of homogeneity criteria. However, most pizza toppings have no homogeneous appearance, so region-based segmentation is not suitable for segmenting a pizza image. The image segmentation method developed by Sun and Brosnan (2003b) is therefore not completely automatic. It is composed of the following steps. First, segmentation of the pizza base from the white background is achieved by setting the RGB (red, green, blue) model values in the range of 0–255. Following this, segmentation of ham and mushroom toppings is performed using the HSI model, by setting the hue, saturation, and intensity in the ranges [168, 23], [6, 45] and [57, 230], respectively. Segmentation of the ham for the combined ham and mushroom area is achieved by setting the hue, saturation, and intensity in the ranges of [250, 8], [15, 50] and [140, 210], respectively. Finally, because some of the mushroom pieces are very white in color, and are not segmented by the HSI model, an automatic edge-detection method is used to extract these areas.


(a) Figure 18.4 2004).

(b)

A pizza topping image: (a) original image; (b) the result of image segmentation (Sun and Du,

To partition a pizza-topping image into homogeneous regions automatically, Sun and Du (2004) developed a new region growing-and-merging method, known as “stick growing and merging” (SGM), which employs the traditional region-based segmentation as the dominant method and combines the strengths of both thresholding and edge-based segmentation techniques. The algorithm consists of four major steps: stick initialization, stick merging, sub-region merging, and boundary modification. It starts with initial decomposition of the image into small sticks and non-sticks. The small sticks are merged to obtain the initial sub-regions on the basis of homogeneity criteria. Then smaller sub-regions, with only one stick, are merged into larger sub-regions, and subsequently all sub-regions are merged into regions according to the criteria. Finally, non-sticks and separate small sticks are merged, and the degree of boundary roughness is reduced by boundary modification. Figure 18.4b shows the segmented results of a pizza-topping image, which includes ham, red and green peppers, shredded cheese, and tomato sauce. The original image (Figure 18.4a) is complex for several reasons, including the inhomogeneous character of the foods, object overlapping, shadows, and light reflection. 4.2.2 Determination of topping distribution

In order to determine the distribution of the topping, Sun (2000) developed a practical method by dividing the pizza image into four equal-area quarters (Q1, Q2, Q3, and Q4) and four equal-area radial sections (C1, C2, C3, and C4), as shown in Figure 18.5. The percentage of the topping in each quarter and section is then compared. A pizza with more even distribution of toppings will have similar topping percentages in each quarter and section. Computer vision is coupled with fuzzy logic analysis to evaluate the quality of pizza toppings based on percentage and distribution (Sun and Brosnan, 2003b). Three indices


Q1

Q2

C3 Q3

C1

C2

C4

Q4

(a)

(b)

Figure 18.5 Determination of topping distribution: (a) four equal-area quarters; (b) four equal-area sections (Sun, 2000).

are chosen for evaluation of the topping quality on the pizzas, which are defined as follows: Ham area percentage (%) (HAAP) = (ham area/base area) × 100% Mushroom area percentage (%) (MAP) = (mushroom area/base area) × 100% Topping area percentage (%) (TAP) = {(ham area + mushroom area)/base area} × 100 percent The results for the areas of the whole pizza, mushroom, and ham toppings can be easily obtained based on the segmented images. From these results, the values of HAAP, MAP, and TAP can be determined, and these are subsequently used for the construction of the fuzzy sets. This involves the establishment of the fuzzy sets “even spread,” “acceptable overwipe,” “reject overwipe,” “acceptable underwipe,” and “reject underwipe” for each of the above indexes. From these, the ambiguous degree (AD) is calculated to determine the most effective and least fuzzy parameter. Using the fuzzy evaluation score (FES), the pizza-topping quality is assigned a rank according to the percentage and distribution of different ingredients. It is reported by Sun and Brosnan (2003b) that the TAP index is the best fuzzy parameter, as it has the smallest AD value and hence displays the least fuzziness. In contrast, HAAP is the weakest parameter, therefore it is found to have the least influence on the fuzzy judgment. According to the FES values, the misclassification rate over the five levels examined is 24 percent. However, when only two quality levels are considered (i.e. acceptable and unacceptable), the resulting accuracy is 100 percent. This is an improved level of accuracy over the 92 percent attained for the pizza sauce spread quality study (Sun and Brosnan, 2003a), as more effective parameters are considered. Also, the characteristics used in this assessment of quality are less fuzzy, and hence a correct outcome is easier to achieve.


5 Conclusions With regard to manufacturing procedures, this chapter has discussed the application of computer vision for pizza quality evaluation. Using computer vision, the production speed and efficiency of pizza can be improved, as well as the accuracy of evaluation, with an accompanying reduction in production costs. The manufacturers can produce consistent, standard products even under variable environmental conditions and with the inherent sensitivity of pizza-making. For pizza base production, the SVM classifiers perform better than other classifiers, with the best classification accuracy of 98.3 percent being achieved when just two levels (acceptable and unacceptable) are considered. As for the multi-classification of pizza bases, the one-versus-one and the DAG method have similar performances. Both fuzzy logic and SVM have been demonstrated to be feasible methods for binary classification of pizza sauce spread with various success. However, multi-classification of pizza sauce spread is not an easy problem, and further research is needed to improve the performance. The visual quality of pizza toppings can be evaluated and classified according to their color information, topping percentage, and distribution by using computer vision techniques. For binary classification of pizza toppings according to color, the polynomial SVM classifier with proper color space transformation selection outperforms the C4.5 classifier and the RBF_NN classifier, while the one-versus-one method is more suitable for multi-classification of pizza toppings. The extraction of topping percentage and distribution depends greatly on the segmentation quality of pizza-topping images. Traditional image segmentation methods are only partly suitable for most pizza images, and more robust approaches should be developed.

Nomenclature n xn

evaluation index fuzzy variable value for index n

Abbreviations: AD CCD DAG FES HAAP HAP HSV MAP MF PCA RBF RGB

ambiguous degree charge-coupled device directed acyclic graph fuzzy evaluation score ham area percentage heavy area percentage hue, saturation, and value mushroom area percentage magnitude factor principal component analysis radial basis function red, green, and blue

References 445

SAP sauce area percentage SGM stick growing and merging SRI spatial ratio I SRII spatial ratio II SVM support vector machine TAP topping area percentage

References Anglemier AF, Montgomery MW (1976) Amino acids, peptides, and proteins. In Principles of Food Science. Part 1. Food Chemistry (Fennema OR, ed.). New York: Marcel Dekker, Inc., pp. 205–284. Anonymous (1994) Pizza producers look to Europe. Food Ireland, 4/5, 7, 9. Bishop CM (1995) Neural Networks for Pattern Recognition. New York: Oxford University Press. Boser BE, Guyon I, Vapnik V (1992) A training algorithm for optimal margin classifiers. In Proceedings of the 5th Annual Workshop on Computational Learning Theory, pp. 144–152. Burg JC (1998) Piecing together the pizza puzzle. Food Product Design, 7 (11), 85–86. Canny J (1986) A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8 (6), 679–698. Crammer K, SingerY (2001) On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2, 265–292. Du C-J, Sun D-W (2004) Shape extraction and classification of pizza base using computer vision. Journal of Food Engineering, 64 (4), 489–496. Du C-J, Sun D-W (2005a) Pizza sauce spread classification using color vision and support vector machines. Journal of Food Engineering, 66 (2), 137–145. Du C-J, Sun D-W (2005b) Comparison of three methods for classification of pizza topping using different color spaces. Journal of Food Engineering, 68 (3), 277–287. Du C-J, Sun D-W (2005c) Multi-classification of pizza using computer vision and support vector machine. Computers and Electronics in Agriculture (submitted). Friedman S (1989) Regularized discriminant analysis. Journal of the American Statistical Association, 84, 165–175. Gray RM (1984) Vector quantization. IEEE ASSP Magazine, 1 (2), 4–29. Guinee TP, O’Callaghan DJ (1997) The use of a simple empirical method for objective quantification of the stretchability of cheese on cooked pizza pies. Journal of Food Engineering, 31 (2), 147–161. Kreßel UH-G (1999) Pairwise classification and support vector machines. In Advances in Kernel Methods: Support Vector Learning (Schölkopf B, Burges CJC, Smola AJ, eds). Cambridge: MIT Press, pp. 255–268. Lehmann TA, Dubois DK (1980) Pizza crust – formulation and processing. Cereal Foods World, 25 (9), 589–592. Mark JLO (1996) Introduction to radial basis function networks. Technical Report, Centre for Cognitive Science, University of Edinburgh, UK. Mathworks (1998) Matlab Reference Guide. The MathWorks, Inc., Natick, MA, USA.


Matz SA (1989) Formulas and Processes for Bakers. Barking: Elsevier Science Publishers Ltd. Meir R, El-Yaniv R, Ben-David S (2000) Localized boosting. In Proceedings of the 13th Annual Conference on Computational Learning Theory. San Francisco: Morgan Kaufmann, pp. 190–199. Mendel JM, Fu KS (1970) Adaptive Learning and Pattern Recognition Systems, Theory and Applications. New York: Academic Press. Ohta YI, Kanade T, Sakai T (1980) Color information for region segmentation. Computer Graphics and Image Processing, 13, 222–241. Panigrahi S, Misra MK, Bern C, Marley S (1995) Background segmentation and dimensional measurement of corn germplasm. Transactions of the ASAE, 38 (1), 291–297. Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In Proceedings of Neural Information Processing Systems. Cambridge: MIT Press, pp. 547–553. Proctor VA, Heidebrecht K (1998) Multi-layered pizza product, and method of making. US Patent No. 5,720,998. Quinlan JR (1993) C4.5: Programs for Machine Learning. San Mateo: Morgan Kauffman Publishers. Schowengerdt RA (1983) Techniques for Image Processing and Classification in Remote Sensing. New York: Academic Press. Sun D-W (2000) Inspecting pizza topping percentage and distribution by a computer vision method. Journal of Food Engineering, 44 (4), 245–249. Sun D-W, Brosnan T (2003a) Pizza quality evaluation using computer vision – part 1. Pizza base and sauce spread. Journal of Food Engineering, 57, 81–89. Sun D-W, Brosnan T (2003b) Pizza quality evaluation using computer vision – part 2. Pizza topping analysis. Journal of Food Engineering, 57, 91–95. Sun D-W, Du CJ (2004) Segmentation of complex food images by stick growing and merging algorithm. Journal of Food Engineering, 61 (1), 17–26. Therrien CW (1989) Decision, Estimation and Classification. Hoboken: John Wiley & Sons. Unklesbay K, Unklesbay N, Keller J, Grandcolas J, (1983) Computerised image analysis of surface browning of pizza shells. Journal of Food Science, 48, 1119–1123. Vapnik V (1998) Statistical Learning Theory. New York: John Wiley & Sons. Wang HH, Sun D-W (2002) Melting characteristics of cheese: analysis of effects of cheese dimensions using image processing techniques. Journal of Food Engineering, 52 (3), 279–284. Wang HH, Sun D-W (2003) Assessment of cheese browning affected by baking conditions using computer vision. Journal of Food Engineering, 56 (4), 339–345. Wang HH, Sun D-W (2004a) Evaluation of the oiling off property of cheese with computer vision: correlation with fat ring test. Journal of Food Engineering, 61 (1), 47–55. Wang HH, Sun D-W (2004b) Evaluation of the oiling off property of cheese with computer vision: influence of cooking conditions and sample dimensions. Journal of Food Engineering, 61 (1), 57–66. Wang W, Kindstedt PS, Gilmore JA, Guo MR (1998) Changes in the composition and meltability of Mozzarella cheese during contact with pizza sauce. Journal of Dairy Science, 81 (3), 609–614.

Quality Evaluation of Cheese Sundaram Gunasekaran Food and Bioprocess Engineering Laboratory, University of Wisconsin-Madison, Madison, WI 53706, USA

1 Introduction Cheese is a fermented milk product. It is believed to have originated in the Middle East several thousand years ago as a way of preserving milk. Now cheese enjoys popularity unrivalled by many other manufactured foods. Maintaining and improving the quality of cheese and cheese products is an ongoing quest for cheese-makers and researchers. However, the term “quality” is too generic, given the wide array of cheeses manufactured, the many intended end-uses, and the eventual consumers. Even the widely accepted quality factors such as stretchability and meltability are either poorly defined or interpreted differently, depending on the end-user and the end-use. Cheese quality evaluation is further confounded not only by the lack of precise understanding of the effect of different biochemical constituents on cheese properties, but also by the lack of suitable methods to characterize objectively many of the quality attributes. In fact, several current methods of cheese quality evaluation are rudimentary and/or rely on the subjective judgment of the tester. The lack of objectivity of the test methods and the inconsistency of the test results has led both industry and academia to develop new methods and devices. In many cases the new methods have been welcome additions, if not as routine tests for industrial use then as useful tools for researchers and industry personnel alike to improve cheese quality and/or to probe hitherto unknown structure– function relationships. In this chapter discussion is limited to those test methods that involve the application of various computer vision methodologies developed for cheese quality evaluation.

2 Cheese quality characteristics Quality characteristics of cheeses can be grouped into different categories – microbial, chemical, physical, functional, etc. However, the quality characteristics of interest for Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


19

448 Quality Evaluation of Cheese

computer vision applications are mostly physical. The computer vision technique is primarily a substitute for human vision, but with several advantages: objectivity, consistency, speed of operation, cost-effectiveness, etc. Furthermore, it can perform better than human vision, in the spectral range outside human perception and in conditions unsafe for human operators. Thus, many of the visually perceptible changes in cheese during manufacturing, storing, and processing have been measured using computer vision.

2.1 End-use qualities 2.1.1 Meltability The melting quality of cheese is commonly referred to in the industry as its “meltability.” Several industrial and academic researchers have interpreted the term differently, often to suit a specific need or application. For example, meltability has been considered as the property of cheese shreds to fuse together upon heating. This definition or description is suitable for applications such as assessment of pizza quality, but is rather difficult to use as a measurement criterion. From an objective measurement perspective, meltability may be defined as “the ease and extent to which cheese will melt and spread upon heating” (Gunasekaran and Ak, 2002). This definition encompasses two aspects: the ease of melting, and the extent of flow. The ease of melting is most directly related to the heat transfer and thermal phase change properties of the cheese; the extent of flow is related to the rheological properties of cheese at high temperatures, as well as the force necessary to cause the flow. Several empirical methods have been developed for cheese meltability measurement; chief among them are the Arnott test (Arnott et al., 1957) and the Schreiber test (Kosikowski, 1977). In these tests the sample, typically a thin disk of cheese, is heated at a preset temperature and for a preset duration (Arnott test, 100◦ C for 15 min; Schreiber test, 232◦ C for 5 min), and the change in sample height (Arnott test) or diameter (Schreiber test) is measured and used as an index of cheese meltability. These tests are illustrated in Figure 19.1. Needless to say, the empirical nature of these tests leads to inconsistent results; there is also a marked lack of correlation between the Schreiber and Arnott test results (Park et al., 1984). Many attempts have been made to improve the results of these empirical tests (Muthukumarappan et al., 1999a; Altan et al., 2005) and to formulate new semi-empirical and fundamental test methods and devices (Ak and Gunasekaran, 1996; Sutheerawattananonda and Bastian, 1998; Wang et al., 1998; Muthukumarappan et al., 1999b; Kuo et al., 2000; Gunasekaran et al., 2002), yet the methods based on the Schreiber test are still the most commonly used in the industry. Since melting of cheese is accompanied by visible changes of size and shape, cheese meltability is a natural target for computer-vision based measurement for improved accuracy and consistency. The current author’s research group at the University of Wisconsin-Madison (Muthukumarappan et al., 1999a) was the first to recommend and measure sample melt spread area, rather than its change in height or diameter, as an index of cheese meltability. This was perhaps the first published account of using computer vision methodology for measuring cheese meltability. Following this, in a series of papers, Wang and Sun (Wang and Sun, 2001; Wang and Sun, 2002a; 2002b)

Cheese quality characteristics 449

(a)

(b) Figure 19.1 (a) The Arnott test measures change in sample height and (b) the Schreiber test measures change in sample diameter to determine cheese meltability.


130

Melting degree (%)

Melting degree (%)

180

160

140

120

120

110

100

100

90 0

2

4

6

8

10 12 14 16 18 20 22 Time (min)

0

2

4

6

8

10 12 14 16 18 20 22 Time (min)

70˚C

100˚C

130˚C

70˚C

100˚C

130˚C

160˚C

180˚C

200˚C

160˚C

180˚C

200˚C

(a)

(b)

Figure 19.2 Melting degree (ratio of cheese area after and before heating): (a) Cheddar and (b) Mozzarella cheese at different temperatures (Wang and Sun, 2002c).

applied a similar procedure to measure cheese spread area upon melting. They used the ratio of spread area or increase in spread area before and after melting of the cheese sample to represent meltability. The melting degree (ratio of cheese area after and before heating) and melting rate (rate of change in melt area during the first minute of heating) were calculated. The melting degree of Cheddar and Mozzarella cheeses measured at different temperatures as a function of heating time is presented in Figure 19.2. Both sample size and test temperature significantly affected the meltability measurements. They reported an optimal temperature range between 140◦ C and 160◦ C for both Cheddar and Mozzarella cheeses. Gunasekaran and colleagues proposed additional changes to the Schreiber test protocol (Gunasekaran et al., 2002). They replaced the convective oven, typically used for the test, by direct conduction heating via the metal plate on which the cheese disk is heated and allowed to flow. This method was not only faster but also better, because not requiring an oven to perform the melt test reduced the overall cost and space requirements. In addition, the sample was more easily accessible for spread length and/or area measurements. The conduction-heating test also allows continuous cheese melt/flow measurement. For example a laser beam or a computer vision system camera can be used to continually record the cheese spread length or area, respectively, for automatic meltability determination (Figure 19.3). Further, this system can be adapted to make multi-sample measurements simultaneously (Figure 19.4). These improvements could enable more consistent cheese meltability measurement, making it faster and more efficient than the currently available methods.


Image output ADC Board Output (analog)

Output (digital)

CCD Camera DC 5 V

PC

AC 70 V

Heat controller

Power supply

Figure 19.3 Schematic of the non-contact cheese flow measurement system using a computer vision system in conjunction with the conduction of heating bottom plate. Associated components for temperature control and cheese melt spread area measurement are also included (Gunasekaran et al., 2002).

Laser sensor Cheese samples

Rotating hot plate Figure 19.4 Multi-sample testing system with laser non-contact cheese meltability sensor. The laser sensor can be replaced by a digital image system camera (Gunasekaran et al., 2002).

2.1.2 Browning and blister formation

Light-brownish discoloration is observed on the cheese surface after cooking cheese and cheese-containing foods. While such mild browning is acceptable and even desirable, excessive browning is undesirable. The discoloration of cheese is the result of the typical Maillard browning reaction that occurs between the reducing sugar lactose or galactose and amino acids (Kosikowksi and Mistry, 1997). About half of all pizza restaurants have reportedly encountered this quality problem (Pilcher and Kindstedt, 1990). The extent of browning is determined either qualitatively by sensory evaluation (Johnson and Olson, 1985; Lelievre et al., 1990; Oberg et al., 1991), or by reflectance colorimetery using a commercial colorimeter (Matzdorf et al., 1994; Mukherjee and Hutkins, 1994; Fife et al., 1996). Wang and Sun (2001, 2003) evaluated the cooked color of cheese using computer vision technology and analyzed the effect of baking conditions on browning. In order to assess the browning property, they extracted the average gray value (GV) of the images of cheese slices heated at different temperatures (70–200◦ C) for various durations (5–30 minutes). The GV is a measure of mean luminance of pixels inside a region of interest; it normally ranges from 0 (dark, pure black) to 255 (bright, pure white). Since the GV of cheese decreases upon cooking due to the

120

180

110

160 Browning factor (%)

Browning factor (%)


100

90

120

100

80 70

140

0

5

10

15 20 Time (min)

25

30

35

80

0

2

4

6

8 10 12 14 16 18 20 22 Time (min)

70˚C

100˚C

130˚C

70˚C

100˚C

130˚C

160˚C

180˚C

200˚C

160˚C

180˚C

200˚C

(a)

(b)

Figure 19.5 Browning factor versus cooking time for (a) Mozzarella cheese and (b) Cheddar cheese at baking temperatures of 70–200◦ C (Wang and Sun, 2003).

cooked color, a browning factor was defined as the ratio between GV before (GV0 ) and after (GVt ) cooking: Browning factor = GV0 /GVt Thus, the higher the browning factor, the greater the extent of the browning of heated cheese. The browning factor values of Cheddar and Mozzarella cheeses as a function of baking time and temperature are presented in Figure 19.5. The more intense browning occurring at elevated temperatures is evidenced by the high browning factor values. Similar results were obtained from industrial processed cheeses (Caric and Kalab, 1982). The Cheddar cheese tended to develop more intense brown discoloration than Mozzarella cheese under similar cooking conditions. The browning factor of Mozzarella cheese was almost proportional to the cooking temperature from 70–200◦ C. For the Cheddar cheese, different changes of browning factor were observed within different temperature ranges. When pizza is baked, it often develops blisters. Figure 19.6 illustrates different levels of blister formation on a pizza. According to Rudan and Barbano (1998), during baking the evaporating water and air is trapped between cheese shreds and collects in bubbles under the melting cheese surface. When this steam and air expand, the thin layer of cheese is lifted off the rest of the pizza, initiating the blister formation. As the cheese rises, the top of the blister becomes thinner, liquid fat at the surface flows down the sides of the forming blister, moisture is lost from cheese at the top surface of the blister, and the top of the blister turns brown. At other locations on the surface of the pizza the cheese retains its white color because the free oil present there prevents excessive


Figure 19.6 Different extents of blister formation on a pizza during baking.

moisture loss from the surface.Yun et al. (1995) employed an image-processing method to characterize blisters that develop on pizza upon baking. The intensity of browning is affected by such baking conditions as time, temperature, reactive amino acids, and carbohydrates in the cheese (Johnson and Olson, 1985). There may be a number of factors affecting size, coverage, and color of the blisters. An image-analysis program was developed for quantifying the number, area of coverage, size, and color intensity of blisters on pizza as affected by the age of Mozzarella cheese. Digital images of pizza baked at 232◦ C for 5 minutes were evaluated. The number of blisters decreased (from 290 to 190) when the cheese was stored from 2 to 8 weeks, although the average area of blisters increased and coverage decreased from 16.5 percent to 12 percent. Blister color was lighter (GV of 98 from 115). 2.1.3 Oiling off

Upon heating, an oily layer is clearly visible on the surface of most cheeses. This is known as “free-oil formation”, “oiling-off”, or “fat leakage.” As these names imply, oiling off is due to the melting of fat globules during heating, and the melted fat eventually leaves the cheese protein matrix structure. The free-oil formation, just like the development of the cooked brown color, is expected and even desirable during the heating of cheese (Rudan and Barbano, 1998). However, excessive oiling-off is undesirable, especially because it presents an unhealthy and greasy aspect in cheesecontaining foods (Kindstedt and Rippe, 1990). The extent of free-oil formation is measured either qualitatively by the fat-ring test (Breene et al., 1964) or quantitatively


(a)

(b)

Figure 19.7 Images of a Cheddar cheese disk, (a) before cooking and (b) after cooking at 110◦ C for 5 minutes. After cooking an oil ring was formed around the cheese disk due to the spreading of free oil in the filter paper.

by a centrifuge test (Kindstedt and Rippe, 1990). In the fat-ring test, disk-shaped cheese samples are placed on a piece of filter paper and heated at 110◦ C for 5 minutes. The area of oily ring formed around the cheese sample is measured and used as an index of free-oil formation (Figure 19.7). Wang and Sun (2004a; 2004b) followed the fat-ring test protocol and used computer vision technology to measure the fat-ring area. This area was correlated (r = 0.852) with free-oil formation determined with the traditional fat-ring test (Figure 19.8). The low correlation is clearly evident by the data scatter around the best-fit line, especially for Cheddar cheese. In addition, using image processing, they extracted several luminance features of the melted cheese; these are listed in Table 19.1. However, none of these luminance features correlated well with the free-oil formation.

2.2 Cheese shred morphology and integrity Cheese is often shredded not only by the consumers but also by the manufacturers. This allows it to melt evenly and to be sprinkled on easily when included in foods (Dubuy, 1980). These attributes enhance the sales and the use of shredded cheese as a food ingredient. Machined cheeses make portion control and/or fill-weight control easy (Andres, 1983). Shredded cheese has captured nearly 25 percent of the cheese market and its share is still growing, obviously due to the increased use in the ready-to-eat and heat-and-serve food categories. Ideally, shredded cheese should be uniformly and precisely cut. Uniform shreds appear appetizing and appealing to the eye for snacks and salads or when used as a garnish. Conventional shredded cheese is of 3.2-mm or 1.6-mm (1/8 or 1/16 ) square cross-section, while newer “fancy” shreds are 0.8-mm or 0.4-mm (1/32 or 1/64 ) in cross-section. Cheese processors often find it difficult to maintain the integrity of cheese shreds, especially when composition and manufacturing parameters vary widely. It is essential


700 600

Fat leakage (%)

500 400 300 200 100 0

C1

C2

C3

C4

C5

M1

M2

M3

M4

Sample (a) 100 90

POA ⫽ 0.0929FL-9.1227 R ⫽ 0.852

Percentage oil area (%)

80 70 60 50 40 30

Mozzarella

20 10

Cheddar

0 ⫺10

0

100

200

300

400

500

600

700

800

900

Fat leakage (%) (b) Figure 19.8 Fat leakage of Cheddar and Mozzarella cheeses as determined with oil ring test: (a) C1–C5, Cheddar cheeses; M1–M4, Mozzarella cheeses; (b) correlation between percentage oil area determined with the computer vision system and fat leakage (Wang and Sun, 2004a).

to ensure that the shreds retain such desirable characteristics during handling, distribution, and storage. However, the shreds often crumble, stick, or mat. Special processes are used to maintain the length of each shred so that its breakage or crumbling is minimized. Microcrystalline cellulose is used to prevent caking or stickiness. In order to assure high quality, shredded cheese manufacturers routinely evaluate the size and shape characteristics of the shreds. This quality assurance test provides them with valuable feedback regarding the appropriateness of the cheese-making, -shredding, and -handling processes, and distribution and storage operations. The current evaluation method entails sieving a sample of shredded cheese to collect fragments that pass through a certain sieve size. This method, while focusing on the


Table 19.1 Luminance features of melted cheese for determining the amount of free-oil formation (Wang and Sun, 2004a). Feature

Description

Number of pixels (N) Mean (µ)

Total number of pixels that fall in the boundary of cheese in an image Average luminance gray value of pixels within the cheese area boundary Standard deviation of a histogram, signifying how widely values are dispersed from the mean Square of SD Skewness of a histogram, representing the degree of asymmetry of a histogram around its mean Measure of the relative peakedness or flatness of a histogram compared with the normal distribution Middle gray level of a luminance histogram The most frequently occurring or repetitive gray value in a histogram Minimum gray value in a histogram at which the number of pixels is more than zero Maximum gray value in a histogram at which the number of pixels is more than zero The difference between Min and Max The product of µ and N

Standard deviation (SD) Variance (Var) Skewness (Skew) Kurtosis A Median Mode Minimum (Min) Maximum (Max) Range of gray values (GV range) Integrated gray value (IGV)

fragmented small pieces, ignores evaluation of the characteristics of the more important “fingers.” High-quality shredded cheese will have individual shreds of uniform size. However, attempting to evaluate individual shred characteristics manually is a tedious and time-consuming task. Therefore, cheese processors need a tool for rapid and objective evaluation of individual shred size and shape characteristics with little human intervention. Computer image-processing techniques have been developed to accomplish this task. Apostolopoulos and Marshall (1994) were the first to apply image-processing methods for characterizing cheese shreds. However, they assumed, unrealistically, that the cheese shreds do not touch or overlap each other. Using manually pre-sorted cheese shreds, they employed computer image analysis to characterize shred shape and size. When objects touch and/or overlap, it presents problems in the product being evaluated. McDonald and Chen (1990) developed a morphological algorithm to separate connected muscle tissues in an image of beef ribeyes. Shatadal et al. (1995) developed an algorithm to separate touching wheat kernels. Image morphology refers to the geometric structure within an image, which includes size, shape, particle distribution, and texture characteristics. For successful image morphology evaluation, one of the first and most important requirements is to understand the characteristics of the object being analyzed – in this case, the cheese shreds. USDA specifies the body and texture of shredded cheese as follows (USDA, 1996): • • •

A cheese shred is a ribbon-shaped object A cheese shred has smooth boundaries A cheese shred can only be curved within the limited flexibility, i.e. an unbroken cheese shred cannot have a sharp-angled corner


•

The width of a single cheese shred should not be greater than two times the preset shred width. Any shreds whose width is greater than two times the preset width is considered to be matted.

Ni and Gunasekaran (1998, 2004) used these as guidelines in developing two algorithms, i.e. the image thinning and skeletonization algorithm and the X–Y sweep algorithm, for evaluating shredded cheese morphology and integrity even when the shreds touched and overlapped one another. 2.2.1 Image thinning and skeletonization algorithm

An image skeleton is a powerful analog concept that may be employed for the analysis and description of shapes in binary images. It plays a central role in the pre-processing of image data. A comprehensive review on thinning methodologies has been presented by Lam et al. (1992). In general, a skeleton may be defined as a connected set of medial lines along the limbs of a figure. The basic idea of the skeleton is to eliminate redundant information while retaining only the topological information concerning the shape and structure of the object that can help with recognition. Thinning is perhaps the simplest approach to skeletonization. It may be defined as the process of systematically stripping away the outermost layers of a figure until only a connected unit-width skeleton remains. A number of algorithms are available to implement this process, with varying degrees of accuracy. The skeleton is supposed to be the path traveled by the pen in drawing out the object. However, this is not always the case. The most reliable means of achieving the straight line would probably be via some high-level interoperation scheme that analyzes the skeleton shape and deduces the ideal shape (Davies, 1997). Naccache and Shinghal (1984) compared the results of fourteen skeletonization algorithms. Thinning algorithms can be divided into two categories: sequential thinning and non-iterative thinning. Sequential thinning examines the contour points for deletion in a pre-defined order. Non-iterative thinning methods are not pixel-based; they produce a certain median or center line of the pattern directly in one pass without examining all individual pixels. Ni and Gunasekaran (1998) have applied a sequential thinning algorithm for evaluating cheese shred morphology when they are touching and overlapping. Since it is pixel-based, the sequential thinning method is rather slow and very sensitive to noise at the boundaries (Lam et al., 1992) – for example, small irregularities at object boundaries may result in misleading image skeletons (Ni and Gunasekaran, 1998). Three example cases considered: single, touching, and overlapping shreds (Figure 19.9a). The image skeletons were obtained, and syntactic graphs representing the shreds were formed. The results of thinning the binary image (Figure 19.9b) are shown in Figures 19.9c to 19.9f. To illustrate the usefulness of the pre-processing steps (dilation and erosion), the result of thinning the original image (without any preprocessing) are shown in Figure 19.9e and 19.9f, respectively. For example, the circle on one of the single shreds in Figure 19.9e is due to the small hole on the corresponding shred image in Figure 19.9a. Manually-measured and computer-vision calculated shred lengths were comparable. The image thinning and skeletonization algorithm performed very well. When compared with the manual shred length measurement, the error was less than 4 percent.


(a)

(b)

(c)

(d)

(e)

(f )

Figure 19.9 Single, touching, and overlapping shredded Mozzarella cheese samples used for image thinning and skeletonizing. (a) digital image, (b) binary image, (c) result after morphological dilation step, (d) result after morphological erosion step, (e) skeleton obtained without dilation and erosion steps, (f) skeleton after using dilation and erosion steps (Ni, 2000).

2.2.2 The X–Y sweep algorithm This algorithm sweeps a visual scene in the X-direction (an X-sweep scans the image row-by-row from left to right to extract the vertical block segments which satisfy some preset criteria) and the Y-direction (a Y-sweepscans the image column-by-column from top to bottom to extract the horizontal block segments which satisfy some preset criteria), and generates two sets of run-length codes. According to the width conditions and spatial relations with the neighbor run-length codes, the run-length codes are grouped as segments. A joint is formed by collecting the pixels that cannot be swept


Vertical segment Ambiguous segment Horizontal segment

Figure 19.10 Vertical, horizontal, and ambiguous (neither horizontal nor vertical) segments identified by X–Y sweep (Ni and Gunasekaran, 2004).

(a) Horizonatal block

(b) Vertical block

Joint

Figure 19.11 (a) A set of cheese shreds arranged to represent different patterns of overlap and (b) results of X–Y sweep (Ni and Gunasekaran, 2004).

through either in the X-direction or the Y-direction. The occluded shred-shaped objects are recovered by merging the neighboring blocks based on the local, semi-local, and global descriptions. The topological sorting method is used to find the best match. The vertical, horizontal, and ambiguous block segments identified by the X–Y sweep are illustrated in Figure 19.10. The X–Y sweep method worked well to identify all shred-shaped objects; an accuracy of 99 percent was obtained for pre-cut touching and overlapping straight copper wires. The tests with “in situ” cheese shreds (i.e. touching and overlapping shreds as poured from the packaging) were about 95 percent accurate in estimating shred lengths. Unlike the thinning and skeletonization algorithm, the X–Y sweep method derives the geometric properties from regional contour information. Therefore, it is insensitive to boundary noise. The robustness of this method was evaluated using images that represent different patterns of overlapping cheese shreds. As shown in Figure 19.11, the algorithm effectively extracted horizontal and vertical segments and joints for different patterns of touching and overlapping cheese shreds. Several additional images of varying patterns of cheese shreds and shred-like objects (e.g. wires) were tested with excellent results (Ni and Gunasekaran, 1998). In general, the X–Y sweep method


1

8

4

2 3

5 6

14 13 27

18 19

17 20

21

28

29

25 26

Horizonatal block Figure 19.12

15 10

11 12 22

24 23

16

9

7

31

30 Vertical block

Joint

Numbered shred segments for the sample in Figure 19.11 (Ni and Gunasekaran, 2004).

correctly detected the primitive segments and joints. The algorithm was very efficient and fairly insensitive to the boundary noise, compared to the sequential thinning method of Ni and Gunasekaran (1998). The robustness of the algorithm was also tested using the same image rotated through 90◦ , 180◦ , and 270◦ . In all cases, the algorithm recognized the same primitive segments. Once the image segments have been demarcated, they are numbered sequentially (Figure 19.12). The local description of each segment determines whether a segment can be merged with another segment based on their local spatial relationships. Generally, the orientation of a block segment can be represented by a curve that connects the middle point of each run. However, since a cheese shred can be curved as its length extends, this orientation representation may be misled (e.g. segment 29 in Figure 19.12). Using the Hough transform as described by Parker (1996) on the image clusters in Figure 19.12, the occluded segments were recovered by merging the segments that have similar parameters in the Hough space. After identifying and extracting the blocks and joints, adjacency graphs were used to represent their topological relations. Since shredded cheese is rather an entangled mass, a procedure was developed to draw a sub-sample suitable for image processing. Cheddar cheese shreds (1.6 × 1.6 mm in cross-section) were poured from the original packaging over a dry paper towel. A wet paper towel was placed over this and gently pressed down to make good contact with the shreds. The wet towel was removed, and the pattern of shreds sticking to the paper towel was used to represent “in situ” shred distribution (Figure 19.13). The segmentation results of the “in situ” cheese shred sample are shown in Figure 19.14. The ability of the X–Y sweep algorithm to generate the cheese shred length distribution histogram could be of significant benefit to cheese processors trying to obtain an objective description regarding shred uniformity. Furthermore, a histogram can also be used to characterize shred disintegration during storage and handling. Besides the shred length distribution, two empirical quality indices were developed based on the X–Y sweep measurements: degree of free-flowing (DOF), and degree of matting (DOM). The DOF can be defined as the ratio of number of clusters (group of two or more cheese shreds) in the sample to the number of terminal blocks: DOF = Number of clusters/Number of terminal blocks


(a)

(b)

Figure 19.13 (a) Cheddar cheese shreds spread over a dry paper towel; (b) a sub-sample is drawn using a wet paper towel placed over the shreds (Ni and Gunasekaran, 2004).

(a)

(b)

(c)

Figure 19.14 (a) Distribution of Cheddar cheese shreds as poured from the packaging; (b) and (c) two of the six “in situ’’ sub-samples drawn and segmented (Ni and Gunasekaran, 2004).

In a sample without any shreds touching and overlapping, DOF = 1. Therefore, the closer this index is to 1, the more easily the shreds can be poured and/or spread. Similarly, the DOM was defined as the ratio of “joints” area to the shreds area: DOM = Size summation of joints/Size summation of all objects The joints represent the touching and/or overlapping regions. The larger this region, the more extensive is the matting. When there is no overlap, DOM = 0 signifying absence of matting. The DOM increases when the degree of matting gets worse. A note of caution: the DOM can be high even if the shreds are not matted – i.e. they are simply touching and/or overlapping due to the sampling procedure. Therefore, a carefully defined sampling procedure should be followed, depending on how the measurements made with the X–Y sweep algorithm will be used. The sample in Figure 19.13 had a DOF value of 0.93 and a DOM value of 0.06. These values indicate that the sample did not have any appreciable level of matting, and could be easily sprinkled on foods such as pizza. It should also be noted that high DOF and low DOM do not necessarily mean the best shred quality. However, these indices along with the shred length histogram would provide a quantitative estimate of the overall shred uniformity and integrity.


(a)

(b)

(c) Figure 19.15 Digital images of calcium lactate crystals on the surface from each of the three samples of smoked Cheddar cheese (a, b, and c) that contained three levels of crystal growth. Crystals appear white on the original images (left), and green after the images were color-thresholded (right) to quantify crystal coverage by image analysis (Rajbhandari and Kindstedt, 2005b).

3 Cheese defects 3.1 Calcium lactate crystals Calcium lactate crystals, when present on the surface of Cheddar cheese, appear as white, crystalline specks (Figure 19.15). Though they are not harmful, consumers misconstrue them as a microbial problem (Tuckey et al., 1938; McDowall and McDowell, 1939; Shock et al., 1948; Farrer and Hollberg, 1960). This perceived quality defect results in economic loss for the cheese-makers (Chou et al., 2003; Swearingen et al., 2004). The formation of calcium lactate crystals has been attributed to the supersaturation of the serum phase of the cheese with calcium and lactate ions, which crystallize and eventually grow into larger aggregates (Dybing et al., 1986; Kubantseva et al., 2004; Swearingen et al., 2004). There are several reports elucidating the mechanism and causative factors of the formation of calcium lactate crystals (Pearce et al., 1973; Sutherland and Jameson, 1981; Johnson et al., 1990a; 1990b; Chou et al., 2003; Swearingen et al., 2004; Rajbhandari and Kindstedt, 2005a).

Cheese defects 463

(a)

(b)

(c)

Figure 19.16 Three slices of Ragusano cheese from different blocks with different levels of gas production: (a) 0.9%, (b) 2.6%, and (c) 6.8% (Caccamo et al., 2004).

The amount of calcium lactate crystals present is qualitatively evaluated by assigning an arbitrary numerical scale ranging from 0 (low) to 4 (extensive) crystal development (Dybing et al., 1986), or 0 to 10 (Johnson et al., 1990a). Rajbhandari and Kindstedt (2005b) evaluated calcium lactate crystals on Cheddar cheese samples using digital photography and image analysis. HSI (hue, saturation, and intensity) color-space thresholding and pixel-counting algorithms were used to identify and calculate the percentage of the total cheese surface area covered by calcium lactate crystals. They also analyzed the calcium lactate crystals for D(−) and L(+) lactate contents by an enzymatic method. A high degree of repeatability was reported for three cheese surfaces, ranging from very slight and geometrically simple to very heavy and geometrically complex crystal coverage, with less than 5 percent error.

3.2 Mechanical openings Distribution of small round holes throughout a cheese block is a characteristic and desirable feature of Emmentaler, Gouda, Ragusano, and Edam cheeses. These holes, known as eyes, are formed primarily from the carbon dioxide produced as propionic acid and citric acid are fermented by the starter organisms, and from nitrogen dissolved in the cheese milk (Akkerman et al., 1989; Polychroniadou, 2001). Holes formed in other cheeses (e.g. Tilsit and Havarti) are not called eyes but are fairly typical of these cheeses (Polychroniadou, 2001). Different amounts of gas holes formed in Raguano cheese are shown in Figure 19.16. The presence of holes in Cheddar-type cheeses, caused by some spoilage organisms producing carbon dioxide, hydrogen and/or hydrogen sulfide, is an indication of a quality defect. Even in cheeses where eyes or holes are expected and accepted, slits or cracks are formed under certain conditions. Generally, gas holes or cracks vary in number, distribution, size, and shape. White et al. (2003) indicated that one of the least controlled defects in round eye cheeses was the development of slits that appeared during refrigerated storage after cheese was removed from the warm room. Slit defects in Cheddar cheese are shown in Figure 19.17. Generally, abnormal gas production in cheese has been divided into two types: early gas production and late gas production (Kosikowksi and Mistry, 1997). Early gas is typically produced


(a)

(b)

Figure 19.17 Cheddar cheese with slit defect: (a) 0.85% of the area as slits and (b) 0.65% of the area as slits (Caccamo et al., 2004).

(a) Figure 19.18

(b)

(a) Gray-scale and (b) original images of a cheese slice with holes (Caccamo et al., 2004).

by coliforms, while late gas production in cheese is typically caused by Clostridium tyrobutyricum. In Cheddar cheese, citrate-fermenting lactobacilli are thought to be the cause of gas production that leads to slit formation (Fox et al., 2000). Biochemical and microbiological aspects of hole formation have been well researched (Akkerman et al., 1989; Zoon and Alleersma, 1996; Polychroniadou, 2001). A nucleus is required for a hole to form. Small air bubbles (nitrogen in milk) attached to curd particles may form as nuclei, along with some impurities and small mechanical openings. The nuclei grow into eyes due to diffusion of carbon dioxide. The size, number and distribution of eyes can be related to the time, quantity, intensity, and rate of carbon dioxide production (Polychroniadou, 2001). Akkerman et al. (1989) discussed the mechanism of eye formation and growth in detail. Caccamo et al. (2004) used computer image analysis to quantify the extent of hole formation in Emmental and Ragusano cheeses, and slit formation in Cheddar cheese. RGB (red, green, and blue) thresholding and pixel-counting algorithms were used to identify the hole areas from the rest of the cheese surface, which served as the background. A typical photograph of an Emmental cheese slice and its gray-scale image are shown in Figure 19.18. Some of the practical issues encountered pertained to the size of the holes and the thickness of the sample slices used for image acquisition. Large holes

Microstructure evaluation 465

(a)

(b)

(c)

Figure 19.19 Photograph of two slices of Emmental cheese of different thickness: (a) 1-mm thick and (b) 10-mm thick; (c) pixel intensity plot of the 1-mm thick slice (Caccamo et al., 2004).

contain shadow regions (Figure 19.16c), and the distribution of holes that run through or partially through the slice thickness also presents problems. These problems made it difficult to determine a uniform threshold value for image segmentation. Figure 19.19 illustrates the loss of hole area that was not through the entire slice thickness. Thus, manual thresholding based on the sample being inspected and the lighting conditions used was recommended.

4 Microstructure evaluation The end-use qualities, such as the texture and elasticity of cheese and other foods, are strongly influenced by their microstructure (Emmons et al., 1980; Stanley, 1987; Ding and Gunasekaran, 1998). Thus, control of food properties for various applications requires a better understanding of the relationships between the food microstructure and macroscopic properties. Microstructural studies are also useful in evaluating the effects of composition and/or technological factors during cheese-making. Image-processing applications in the study of cheese microstructure range from simple two-dimensional (2D) analysis of micrographs obtained using scanning electron microscope (SEM) to in situ three-dimensional (3D) and dynamic four-dimensional (4D) analyses of images reconstructed using 2D optical image slices obtained with confocal laser scanning microscope (CLSM).

4.1 Analysis of SEM micrographs Typically, scanning electron microscopy (SEM) and transmission electron microscopy (TEM) are used for food microstructure evaluation. These traditional techniques, though powerful in terms of yielding high image resolution, do not lend themselves directly for digital image analysis. However, SEM and TEM micrographs can be digitized to quantify the image features. Pastorino et al. (2003) followed this procedure to determine the effect of salt on cheese microstructure. The salt content affects the structural and functional properties of cheese; a high salt content promotes solubilization of caseins (Guo and Kindstedt, 1995; Guo et al., 1997), causing the protein matrix


(a)

(b)

Figure 19.20 Scanning electron micrographs of Muenster cheese after 40 days of storage at 4◦ C: (a) unsalted cheese (uninjected); (b) salt-injected cheese (five injections). Bar = 10 µm (Pastorino et al., 2003).

to become more hydrated and swollen (Guo and Kindstedt, 1995; Guo et al., 1997; Paulson et al., 1998). Adding salt to cheese also affects the cheese composition by influencing the microbial activity (Thomas and Pearce, 1981; Schroeder et al., 1988). In addition, the salt content may also affect cheese proteolysis via microbial and enzyme activities, with high salt levels decreasing the rate and/or extent of proteolysis (Fox and Walley, 1971; Schroeder et al., 1988; Mistry and Kasperson, 1998). Consequently, the salt content may affect both cheese pH and proteolysis, which in turn affect cheese functionalities. Pastorino et al. (2003) took unsalted 4-day-old Muenster cheese blocks and injected salt solution between one and five times to obtain different salt contents ranging from 0.1 percent to 2.7 percent. SEM pictures of the cheese samples are shown in Figure 19.20. The control and no-salt cheese had a structure typical of a stirred/pressed-curd cheese, with protein matrix interspersed by areas that originally contained fat and/or serum (Figure 19.20a). The structure of the salt-injected cheeses looked similar to that of the control cheese, with fat/serum pockets ranging in size between 1 and 11 µm in diameter or length observed throughout the cheese matrix (Figure 19.20b). These micrographs were digitized into gray-scale images and analyzed. In the original digital images, dark pixels corresponded to areas of the micrograph occupied by pockets that originally contained fat and/or serum, while light pixels corresponded to areas occupied by protein matrix. The proportions of black and white pixels, and the areas occupied by them, were then determined by applying the histogram function. In these images (Figure 19.21), fat/serum pockets were clearly differentiated from the protein matrix. Thus, the areas of cheese matrix occupied by fat/serum pockets (dark areas) and protein matrix (light areas) were determined. In the control cheese, the protein matrix occupied 84 percent of the cheese matrix, with fat/serum pockets occupying the remaining 16 percent (Figure 19.21a). Although only significant at P < 0.1, cheese with 2.7 percent salt had 4 percent more protein matrix area than did the control cheese (Figure 19.21b). This is in agreement with the results of Paulson et al. (1998), who observed that salted non-fat Mozzarella has a more homogeneous cheese matrix, with an increased area occupied by protein matrix, compared with unsalted cheese.


(a)

(b)

Figure 19.21 (a) and (b) binary images of scanning electron micrographs in Figures 19.20(a) and (b), respectively, after thresholding. Bar = 10 µm (Pastorino et al., 2003).

4.2 Three-dimensional cheese microstructure evaluation using CLSM Confocal laser scanning microscopy (CLSM) allows the acquisition of high-quality optical images of sections, free from out-of-focus blur or fluorescence flare. Unlike electron microscopy (EM) techniques, CLSM does not require sample fixation and/or dehydration. In addition, when combined with 3D reconstruction techniques, optical sectioning may be sufficient to reveal information typically unobtainable via traditional 2D micrographs (Ogawa et al., 2001). Three-dimensional image processing and reconstruction have been used as new tools for evaluating various aspects of foods (Kalab et al., 1995; Yun et al., 1995). The 3D analysis offers a better understanding of the structure–function relationships of several systems at various stages during processing (Hamberg et al., 2001). In addition, time-resolved 3D microstructural analysis will allow investigation of 4D dynamic changes in food microstructures (Hell et al., 1993; Olsson et al., 2002a, 2002b). An example of sequential 2D slices obtained is illustrated in Figure 19.22. The current author’s research team at the University of Wisconsin-Madison was the first to publish 3D image analysis of in situ cheese microstructure (Everett et al., 1995; Ding and Gunasekaran, 1998; Gunasekaran and Ding, 1999). Using an MRC-600 confocal microscope (Bio-Rad Microscience Limited, Hercules, CA) and a 568-nm krypton/argon laser light source, they obtained 81 sequential optical image slices with interlayer separation distance of 0.5 µm. The samples were stained with 0.1% Rhodamine B to improve image contrast and allow easy observation of fat globules. Several layered images (from top to bottom) of cheeses of different fat contents are presented in Figure 19.23. With an appropriate image reconstruction algorithm, a 3D view was generated as shown in Figure 19.24. Such a reconstructed image allows viewing of the microstructural details of the same sample from different angles, which is not possible with electron microscopy (Figure 19.25). One problem that they faced was chopping off the fat globules at the image edges during sample preparation. Several image features can be extracted and, more importantly, quantified. For example, the number of globules, their size (diameter and volume), and the shape index


(a)

(b)

Figure 19.22 (a) A two-dimensional view and (b) sequential layered two-dimensional optical slicing of cheese microstructure. The dark areas represent fat globules in the cheese (Ding, 1995).

Layer 16

Layer 32

Layer 48

Layer 64

Layer 48

Layer 64

Layer 48

Layer 64

(a)

Layer 16

Layer 32 (b)

Layer 16

Layer 32 (c)

Figure 19.23 Two-dimensional layered images of 1-month-old Cheddar cheeses (the width of each microscopic image is 77 µm). (a) Top view, layered images of the full-fat cheese; (b) Top view, layered images of the low-fat cheese; (c) Top view, layered images of the very-low-fat cheese (Gunasekaran and Ding, 1999).


Figure 19.24 Three-dimensional reconstructed image of fat globules in low-fat Cheddar cheese. Boundary-chopped globules are seen at the image boundaries. Each side of the image is 77 µm (Gunasekaran and Ding, 1999).

Layer 16

Layer 32

Layer 48

Layer 64

Layer 91

Layer 121

Layer 91

Layer 121

(a) Top view

Layer 30

Layer 60 (b) Side view

Layer 30

Layer 60 (c) Front view

Figure 19.25 Top, side and front views of two-dimensional layered images of a three-dimensional image from 2-day old low-fat cheese (the width of each microscopic image is 77 µm) (Ding, 1995).


Globule number (%)

50 40 30 20 10 0

Globule volume (%)

1

3

5

7

9 11 13 14 Diameter (µm)

5

7

9

17

19

16 14 12 10 8 6 4 2 0 1

3

11 13

15

17

19

Diameter (µm)

Globule number (%)

38.8

40.5

18.8

20

10

1.6 0

0.1

0.2

0.3

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Sphericity Figure 19.26 Measurement of three-dimensional characteristics (number, volume, and sphericity) of fat globules in cheese (filled bars are whole globules and open bars are edge-chopped globules) (Gunasekaran and Ding, 1999).

(e.g. sphericity) were computed (Figure 19.26). Thus, for the first time, an objective analysis of cheese microstructure was made possible.

4.3 Dynamic 4D microstructure evaluation Practical difficulties in studying CLSM images arise from errors and distortions in the acquired images, especially under dynamic conditions. Aberrations in the optical path of the specimen significantly affect the position and shape of the confocal point spread


(Carlsson, 1991; Visser et al., 1991; Liljeborg et al., 1994; Hell and Stelzer, 1995; White et al., 1996). Increased scattering is associated with local refractive index inhomogeneities in the specimen, and cause shape misrepresentation. Uneven distribution of intensity in CLSM images is also a major contributor to image distortion. The uneven intensity distribution is generated by dust on the lens, a slanted cover slip, non-uniform camera response, and/or bright spots, which make some parts of the image darker or brighter than others. Optical sectioning through thick samples leads to fluorescence signal attenuation with depth (White et al., 1996). The problems arising from imaging thick biological tissue have been addressed in detail (Durr et al., 1989). Owing to the thickness of the sample, the excited and emitted light are both scattered and absorbed in the sample. During acquisition of optical section images, the confocal point spread can deviate and become distorted due to specimen deformation (Baba et al., 1993; Rudan et al., 1999). Therefore, the Z-axis arrangement of multiple images may be distorted according to the working distance of the objective lens. In addition, the boundary of imaging components tends to be quite irregular, and it may be difficult to distinguish it from the background. Ko and Gunasekaran (2006) presented the error correction procedures used for dynamic in situ analysis of cheese microstructure during melting. They used processed and Mozzarella cheese samples cut into 10 × 10 × 1-mm specimens, and stained them with 1% Nile red solution to facilitate easy visualization of the fat phase during imaging. A CLSM (Biorad MRC 1024, Bio-Rad Inc., UK) attached to an inverted camera (Eclipse TE300, Nikon Inc., Japan) was used. Two-dimensional image layers of 512 × 512 pixel resolution were acquired at 0.5-µm interlayer gaps for a sample depth of 20 µm. Thus, a total of 41 two-dimensional layered images were acquired. CLSM images were obtained continually at each 5◦ C increment in sample temperature, from 25◦ to 70◦ C. The images were corrected for image aberrations due to refractive index mismatch, light attenuation with sample depth, uneven image light intensity across the image area, and image misalignment, etc. An example of such a correction is illustrated in Figure 19.27. After error correction, the 2D image layers were reconstructed into a composite 3D image using commercial 3D reconstruction software (VoxBlast 3.0, Vaytek Inc., Fairfield, IA). Figure 19.28 shows the 3D reconstructed image of sample food systems studied. The 3D microstructures were projected at x = 250◦ , y = 330◦ , and z = 40◦ for easy visualization of their in situ structure. Figures 19.28a and 19.28c show the 3D images reconstructed using uncorrected 2D image layers. The 3D images reconstructed using the error-corrected 2D image layers are shown in Figures 19.28b and 19.28d. As can be seen, the error correction substantially improved the 3D microstructural details. In addition, repeated 3D sampling [4D (x,y,z,t)] was used to reconstruct a 4D image showing dynamic changes in fat-globule microstructure during heating. Figure 19.29 shows changes in the fat globule size and shape during heating of the cheese. As the cheese is heated from 25◦ C to 60◦ C, the number of globules decreases due to agglomeration of the melting fat globules. Furthermore, the globules distort in shape, as indicated by an increase in the average hydraulic radius.


50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

50 ␮m

(a)

(b)

(c)

Figure 19.27 Two-dimensional CLSM images of process cheese at various layer depths: (a) first layer; (b) thirty-first layer; (c) forty-first layer after different image processing steps. Top row, before error correction; second row, after intensity compensation; third row, after unevenness correction; fourth row, after image segmentation.


(a)

(b)

(c)

(d)

Figure 19.28 Three-dimensional reconstructed microstructure of processed cheese, Mozzarella cheese, and BLG gel without (left) and with (right) error corrections. (a) Uncorrected processed cheese at 25◦ C; (b) processed cheese at 25◦ C with error corrections; (c) uncorrected Mozzarella cheese at 30◦ C; (d) mozzarella cheese at 35◦ C with error corrections (Ko and Gunasekaran, 2006).

1200

Number of fat globules

1000

800

600

Before melting at 25˚C After melting at 60˚C Plot 1 regr

400 0

2

4

6

8

10

12

14

16

18

20

Layers (a) Figure 19.29 Dynamic changes in fat globules during heating of cheese: (a) number of globules; (b) average hydraulic radius (Ko, 2005).

22


2.0

Average hydraulic radius

1.8

1.6

1.4

Before melting at 25˚C After melting at 60˚C Plot 1 regr

1.2

1.0 0

2

4

6

8

10

12

14

16

18

20

22

Layers (b) Figure 19.29

(Continued)

5 Conclusions Computer vision systems have provided an enabling technology to add objectivity to several quality-control tasks in the cheese industry. Over the past decade, various computer-vision based systems have been developed to determine different quality factors. The primary criterion has been the visible change in size, shape, color, etc., of the sample being examined. These applications have ranged from routine examination of obvious surface features (e.g. holes) to dynamic analysis of evolving microstructure during processing (e.g. melting of cheese). Despite these developments, computer-vision based measurement systems and control technology have not made any significant inroads in the cheese industry, which, as in other food industries, tends to lag in adopting new technology. During the next decade, with improved processing speed and accuracy, it is likely that computer-vision based quality control will be widely adopted for various routine quality control tasks in the cheese industry.

References Ak MM, Gunasekaran S (1996) Evaluating rheological properties of mozzarella cheese by the squeezing flow method. Journal of Texture Studies, 26 (6), 695–711. Akkerman JC, Walstra P, van Dijk HJM (1989) Holes in Dutch-type cheese. 1. Conditions allowing eye formation. Netherlands Milk and Dairy Journal, 43, 453–476.

References 475

Altan A, Turhan M, Gunasekaran S (2005) Short communication: comparison of covered and uncovered Schreiber test for cheese meltability evaluation. Journal of Dairy Science, 88 (3), 857–861. Andres C (1983) Natural cheese in new form provides improved aesthetic and functional qualities. Food Processing Industry, 44 (12), 64–66. Apostolopoulos C, Marshall RJ (1994) A quantitative method for the determination of shreddability of cheese. Journal of Food Quality, 17 (2), 115–128. Arnott DR, Morris HA, Combs WB (1957) Effect of certain chemical factors on the melting quality of process cheese. Journal of Dairy Science, 40, 957–963. Baba N, Satoh H, Nakamura S (1993) Serial section image reconstruction by voxel processing. Bioimages, 1, 105. Breene WM, Price WV, Ernstrom CA (1964) Manufacture of pizza cheese without starter. Journal of Dairy Science, 47, 1173–1180. Caccamo M, Melilli C, Barbano DM, Portelli G, Marino G, Licitra G (2004) Measurement of gas holes and mechanical openness in cheese by image analysis. Journal of Dairy Science, 87, 739–748. Caric M, Kalab M (1982) Processed cheese products. In Cheese: Chemistry, Physics and Microbiology, Vol. 2 (Fox PF, ed). London: Elsevier, p. 377. Carlsson K (1991) The influence of specimen refractive index, detector signal integration, and non-uniform scan speed on the imaging properties in confocal microscopy. Journal of Microscopy, 163, 167–178. Chou YE, Edwards CG, Luedecke LO, Bates MP, Clark S (2003) Nonstarter lactic acid bacteria and aging temperature affect calcium lactate crystallization in Cheddar cheese. Journal of Dairy Science, 86 (8), 2516–2524. Davies ER (1997) Machine Vision, 2nd edn. New York: Academic Press. Ding K (1995) Food Shape and Microstructure Evaluation with ComputerVision. Madison: University of Wisconsin-Madison. Ding K, Gunasekaran S (1998) Three-dimensional image reconstruction procedure for food microstructure evaluation. Artificial Intelligence Review, 12 (1–3), 245–262. Dubuy MM (1980) The French art of shredding cheese. Food Processing Industry, 49, 52–53. Durr R, Peterhans E, Heydt Rvd (1989) Correction of distorted image pairs with elastic models. European Journal of Cell Biology, 48 (Suppl. 25), 85. Dybing ST, Brudvig SA, Wiegand JA, Huang EA (1986) A simple method for estimating the extent of surface crystal development on colored Cheddar cheese. Journal of Food Protection, 49 (6), 421–422. Emmons DB, Kalab M, Larmond E, Lowrie RJ (1980) Milk gel structure. 10. Texture and microstructure in Cheddar cheese made from whole milk and from homogenized low fat milk. Journal of Texture Studies, 11, 15–34. Everett DW, Ding K, Olson NF, Gunasekaran S (1995)Applications of confocal microscopy to fat globule structure in cheese. In Chemistry of Structure/Function Relationships in Cheese (Malin EL, Tunick MH, eds). New York: Plenum Publishing, pp. 321–330. Farrer KTH, Hollberg WCJ (1960) Calcium lactate on rindless cheese. Australian Journal of Dairy Technology, 15, 151–152.


Fife RL, McMahon DJ, Oberg CJ (1996) Functionality of low fat Mozzarella cheese. Journal of Dairy Science, 79 (11), 1903–1910. Fox PF, Walley BF (1971) Influence of sodium chloride on proteolysis of casein by rennet and by pepsin. Journal of Dairy Research, 38 (2), 165–169. Fox PF, Guinee, TO, Cogan TM, McSweeney PLH (2000) Fundamentals of Cheese Science. Gaithersburg: Aspen Publishers Inc. Gunasekaran S (2006) Automation of food processing. In Food Engineering. Encyclopedia of Life Support Systems (Barbosa-Canovas GV, ed.). Paris: UNESCO Publishing, pp. 745–757. Gunasekaran S, Ak MM (2002) Cheese Rheology and Texture. Boca Raton: CRC Press. Gunasekaran S, Ding K (1999) Three-dimensional characteristics of fat globules in Cheddar cheese. Journal of Dairy Science, 82 (9), 1890–1896. Gunasekaran S, Hwang CH, Ko S (2002) Cheese melt/flow measurement methods: recent developments. Australian Journal of Dairy Technology, 57 (2), 128–133. Guo MR, Kindstedt PS (1995) Age-related changes in the water phase of Mozzarella cheese. Journal of Dairy Science, 78 (10), 2099–2107. Guo MR, Gilmore JA, Kindstedt PS (1997) Effect of sodium chloride on the serum phase of Mozzarella cheese. Journal of Dairy Science, 80 (12), 3092–3098. Hamberg L, Walkenstrom P, Stading M, Hermansson AM (2001) Aggregation, viscosity measurements and direct observation of protein coated latex particles under shear. Food Hydrocolloids, 15 (2), 139–151. Hell SW, Stelzer EHK (1995) Lens aberrations in confocal fluorescence microscopy. In Handbook of Biological Confocal Microscopy (Pawley JB, ed.). New York: Plenum Publishing, pp. 347–354. Hell SW, Reiner G, Cremer C, Stelzer EHK (1993) Aberrations in confocal fluorescence microscopy induced by mismatches in the refractive index. Journal of Microscopy, 169, 391–405. Johnson ME, Olson NF (1985) Nonenzymatic browning of Mozzarella cheese. Journal of Dairy Science, 68 (12), 3143–3147. Johnson ME, Riesterer BA, Chen C, Tricomi B, Olson NF (1990a) Effect of packaging and storage conditions on calcium lactate crystallization on the surface of Cheddar cheese. Journal of Dairy Science, 73 (11), 3033–3041. Johnson ME, Riesterer BA, Olson NF (1990b) Influence of nonstarter bacteria on calcium lactate crystallization on the surface of Cheddar cheese. Journal of Dairy Science, 73 (5), 1145–1149. Kalab M, Allan-Wojtas P, Miller SS (1995) Microscopy and other imaging techniques in food structure analysis. Trends in Food Science and Technology, 6, 177–186. Kindstedt PS, Rippe JK (1990) Rapid quantitative test for free oil (oiling off) in melted Mozzarella cheese. Journal of Dairy Science, 73 (4), 867–873. Ko S (2005) In situ Microstructure and Gelation Properties of Heat-induced Betalactoglobulin Gels. Madison: University of Wisconsin-Madison. Ko S and Gunasekaran S (2006) Error correction of confocal microscopy images for in situ food microstructure evaluation. Journal of Food Engineering, 79 (3), 935–944. Kosikowski FV (1977) Cheese and Fermented Milk Foods, 2nd edn. Ann Arbor: Edwards Bros. Inc., pp. 404–406.

References 477

Kosikowksi FV, Mistry VV (1997) Cheese and Fermented Milk Foods, Vol. 1. Westport: FV Kosikowski LLC, pp. 252–265. Kubantseva N, Hartel RW, Swearingen PA (2004) Factors affecting solubility of calcium lactate in aqueous solutions. Journal of Dairy Science, 87 (4), 863–867. Kuo MI, Wang YC, Gunasekaran S (2000) A viscoelasticity index for cheese meltability evaluation. Journal of Dairy Science, 83 (3), 412–417. Lam L, Lee SW, Suen CY (1992) Thinning methodologies – a comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14 (9), 869–885. Lelievre J, Shaker RR, Taylor MW (1990) The role of homogenization in the manufacture of Halloumi and Mozzarella cheese from recombined milk. Journal of the Society of Dairy Technology, 43 (1), 21–24. Liljeborg A, Czader M, Porwit A (1994) A method to compensate for light attenuation with depth in 3D DNA image cytometry using a CSLM. Journal of Microscopy, 177 (2), 108–114. Matzdorf B, Cuppett SL, Keeler L, Hutkins RW (1994) Browning of Mozzarella cheese during high-temperature pizza baking. Journal of Dairy Science, 77 (10), 2850–2853. McDonald T, Chen YR (1990) Application of morphological image processing in agriculture. Transactions of the ASAE, 33 (4), 1345–1352. McDowall FH, McDowell AKR (1939) The white particles in mature Cheddar cheese. Journal of Dairy Research, 10, 118–119. Mistry VV, Kasperson KM (1998) Influence of salt on the quality of reduced fat Cheddar cheese. Journal of Dairy Science, 81 (5), 1214–1221. Mukherjee KK, Hutkins RW (1994) Isolation of galactose-fermenting thermophilic cultures and their use in the manufacture of low browning Mozzarella cheese. Journal of Dairy Science, 77, 2839–2849. Muthukumarappan K, Wang YC, Gunasekaran S (1999a) Estimating softening point of cheeses. Journal of Dairy Science, 82 (11), 2280–2286. Muthukumarappan K, Wang YC, Gunasekaran S (1999b) Short communication: modified Schreiber test for evaluation of Mozzarella cheese meltability. Journal of Dairy Science, 82 (6), 1068–1071. Naccache NJ, Shinghal R (1984) An investigation into the skeletonization approach of Hilditch. Pattern Recognition, 17 (3), 279–284. Ni H (2000) Evaluating Cheese Shred Morphology and Integrity using Computer Vision. Madison: University of Wisconsin-Madison. Ni H, Gunasekaran S (1998) A computer vision method for determining length of cheese shreds. Artificial Intelligence Review, 12 (1–3), 27–37. Ni H, Gunasekaran S (2004) Image processing algorithm for cheese shred evaluation. Journal of Food Engineering, 61 (1), 37–45. Oberg CJ, Wang A, Moyes LV, Brown RJ, Richardson GH (1991) Effects of proteolytic activity of thermolactic cultures on physical properties of Mozzarella cheese. Journal of Dairy Science, 74 (2), 389–397. OgawaY, Sugiyama J, Kuensting H, Ohtani T, Hagiwara S, Liu X, Kokubo M,Yamamoto A, Kudoh K, Higuchi T (2001)Advanced technique for three-dimensional visualization of compound distributions in a rice kernel. Journal of Agricultural and Food Chemistry, 49, 736–740.


Olsson C, Langton M, Hermansson AM (2002a) Dynamic measurements of betalactoglobulin structures during aggregation, gel formation and gel break-up in mixed biopolymer systems. Food Hydrocolloids, 16 (5), 477–488. Olsson C, Langton M, Hermansson AM (2002b) Microstructures of beta-lactoglobulin/ amylopectin gels on different length scales and their significance for rheological properties. Food Hydrocolloids, 16 (2), 111–126. Park J, Rosenau JR, Peleg M (1984) Comparison of 4 procedures of cheese meltability evaluation. Journal of Food Science, 49 (4), 1158–1161. Parker JR (1996) Algorithm for Image Processing and Computer Vision. Hoboken: John Wiley & Sons. Pastorino AJ, Hansen CL, McMahon DJ (2003) Effect of salt on structure-function relationships of cheese. Journal of Dairy Science, 86 (1), 60–69. Paulson BM, McMahon DJ, Oberg CJ (1998) Influence of sodium chloride on appearance, functionality, and protein arrangements in nonfat Mozzarella cheese. Journal of Dairy Science, 81 (8), 2053–2064. Pearce KN, Creamer LK, Gilles J (1973) Calcium lactate deposits on rindless Cheddar cheese. New Zealand Journal of Dairy Science and Technology, 8, 3–7. Pilcher SW, Kindstedt PS (1990) Survey of Mozzarella cheese quality at restaurant end use. Journal of Dairy Science, 73 (6), 1644–1647. Polychroniadou A (2001) Eyes in cheese: a concise review. Milchwissenschaft, 56 (2), 74–77. Rajbhandari P, Kindstedt PS (2005a) Compositional factors associated with calcium lactate crystallization in smoked Cheddar cheese. Journal of Dairy Science, 88 (11), 3737–3744. Rajbhandari P, Kindstedt PS (2005b) Development and application of image analysis to quantify calcium lactate crystals on the surface of smoked Cheddar cheese. Journal of Dairy Science, 88 (12), 4157–4164. Rudan MA, Barbano DM (1998) A model of Mozzarella cheese melting and browning during pizza baking. Journal of Dairy Science, 81 (8), 2312–2319. Rudan MA, Barbano DM, Yun JJ, Kindstedt PS (1999) Effect of fat reduction on chemical composition, proteolysis, functionality, and yield of Mozzarella cheese. Journal of Dairy Science, 82, 661–672. Schroeder CL, Bodyfelt FW, Wyatt CJ, Mcdaniel MR (1988) Reduction of sodium chloride in Cheddar cheese – effect on sensory, microbiological, and chemical properties. Journal of Dairy Science, 71 (8), 2010–2020. Shatadal P, Jayas DS, Hehn JL, Bulley NR (1995) Seed classification using machine vision. Canadian Agricultural Engineering, 37 (3), 163–167. Shock AA, Harper WJ, Swanson AM, Sommer HH (1948) What’s in those “white specks” on Cheddar? Wisconsin Agricultural Experiment Station, University of WisconsinMadison Bulletin, 474. Stanley DW (1987) Food texture and microstructure. In Food Texture (Moskowitz HR, ed.). New York: Marcel Dekker, Inc. Sutheerawattananonda M, Bastian ED (1998) Monitoring process cheese meltability using dynamic stress rheometry. Journal of Texture Studies, 29 (2), 169–183.

References 479

Sutherland BJ, Jameson GW (1981) Composition of hard cheese manufactured by ultrafiltration. Australian Journal of Dairy Technology, 36 (4), 136–143. Swearingen PA, Adams DE, Lensmire TL (2004) Factors affecting calcium lactate and liquid expulsion defects in Cheddar cheese. Journal of Dairy Science (Abstracts), 87, 574–582. Thomas TD, Pearce KN (1981) Influence of salt on lactose fermentation and proteolysis in Cheddar cheese. New Zealand Journal of Dairy Science and Technology, 16 (3), 253–259. Tuckey SL, Ruehe HA, Clark GL (1938) X-ray diffraction analysis of white specks in Cheddar cheese. Journal of Dairy Science (Abstracts), 21, 161. USDA (1996) Specifications for Shredded Cheddar Cheese. Washington, DC: United States Department of Agriculture, 6 May. Visser TD, Groen FCA, Brakenhoff GJ (1991) Absorption and scattering correction in fluorescence confocal microscopy. Journal of Microscopy, 163, 189–200. Wang HH, Sun DW (2001) Evaluation of the functional properties of Cheddar cheese using a computer vision method. Journal of Food Engineering, 49 (1), 49–53. Wang HH, Sun DW (2002a) Correlation between cheese meltability determined with a computer vision method and with Arnott and Schreiber tests. Journal of Food Science, 67 (2), 745–749. Wang HH, Sun DW (2002b) Melting characteristics of cheese: analysis of effect of cheese dimensions using computer vision techniques. Journal of Food Engineering, 52 (3), 279–284. Wang HH, Sun DW (2002c) Melting characteristics of cheese: analysis of effects of cooking conditions using computer vision technology. Journal of Food Engineering, 51 (4), 305–310. Wang HH, Sun DW (2003) Assessment of cheese browning affected by baking conditions using computer vision. Journal of Food Engineering, 56 (4), 339–345. Wang HH, Sun DW (2004a) Evaluation of the oiling off property of cheese with computer vision: correlation with fat ring test. Journal of Food Engineering, 61 (1), 47–55. Wang HH, Sun DW (2004b) Evaluation of the oiling off property of cheese with computer vision: influence of cooking conditions and sample dimensions. Journal of Food Engineering, 61 (1), 57–66. Wang YC, Muthukumarappan K, Ak MM, Gunasekaran S (1998) A device for evaluating melt/flow characteristics of cheeses. Journal of Texture Studies, 29 (1), 43–55. White NS, Errington RJ, Fricker MD, Wood JL (1996) Aberration control in quantitative imaging of botanical specimens by multidimensional fluorescence microscopy. Journal of Microscopy, 181, 99–116. White SR, Broadbent JR, Oberg CJ, McMahon DJ (2003) Effect of Lactobacillus helveticus and Propionibacterium freudenrichii ssp shermanii combinations on propensity for split defect in Swiss cheese. Journal of Dairy Science, 86, 719–727. Yun JJ, Barbano DM, Bond EF, Kalab M (1995) Image analysis method as quality control and research tool for characterizing pizza blisters. Scanning, 95 (17, Suppl. V), V143. Zoon P, Alleersma D (1996) Eye and crack formation in cheese by carbon dioxide from decarboxylation of gluconic acid. Netherlands Milk and Dairy Journal, 50 (2), 309–318.

Quality Evaluation of Bakery Products Mohd. Zaid Abdullah School of Electrical and Electronic Engineering, Engineering Campus, Universiti Sains Malaysia, 14300 Penang, Malaysia

1 Introduction Cereal products, especially bread, have been a major source of food for the human race since the commencement of civilization (Kent-Jones and Amos, 1967). Bread consumption has increased with the passage of time, such that it has become an integral and established staple part of the diet of the populace. According to the EuroAisa Bakery Report 2005, the Euro-Asian market for bakery products amounted to 60 million tonnes in 2004, worth approximately 126 billion euros, with retail sales registered showing an overall increase of 2.5 percent. The bakery sector is also a major contributor to economic growth and employment opportunities. In Europe there are over 120 000 enterprises active in bread-making, the vast majority of which are small craft bakers. Given its size, the bakery sector can be considered to be one of the most important sectors of the food industry as a whole. However, the bakery products sector is beginning to show signs of maturity, reflecting the fact that sales in 2005 remained sluggish. Intense competition within the bakery sector, combined with evolving consumer expectations, particularly regarding health and convenience, has continued to influence the market trend. Today’s consumers are increasingly demanding improvements in manufactured products. Important characteristic requirements of products include tastiness, crunchiness, a fresh appearance, healthiness, a longer shelf life, convenience, and, of course, a lower price. In the baking industry, ingredients work together to create the desired structure and texture of quality bakery products. Each component affects others, and if they are used at incorrect levels they can destroy the product quality. Moreover, baking requires strict compliance to scaling guidelines, production times, and temperatures. Professional bakers need to blend the exact quantities of each ingredient under ideal conditions to achieve the highest-quality bakery products. However, ideal production conditions and ingredient proportions differ for nearly every bakery item. On the other hand, the ability to manufacture high-quality products is consistently the basis for success in the very competitive industry. As both plant throughput and consumer demands have increased, the pressure on food processors with regard to innovation and Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


20

482 Quality Evaluation of Bakery Products

efficiency to meet product criteria within the bounds and limitations of international and regional standards has also increased. Maintaining product quality and improving process versatility are two important challenges facing bakery manufacturers in the twenty-first century. Consequently, quality assurance and control is becoming an increasingly important goal in the bakery industry. The application of state-of-the-art technology, from processing down to the proportioning of ingredients, enhances the baker’s ability to produce a quality product and reduce material waste. The movement towards automation in the bakery industry reflects the industry’s goal of producing quality products while simultaneously preparing to meet the competition of tomorrow. Like many other food products, customers expect to find constant product quality for the same brand of bakery product. Previous research has shown that visual stimulus is a major factor that influences the consumer judgment of food products (Dubose and Cardello, 1980; Christensen, 1983). From the consumers’ viewpoint, color, size, and shape are three important quality attributes of bakery products. These physical attributes generate memories of past eating experiences and influence the judgment of consumers. The main reason for this is that food is appreciated via its organoleptic properties; and vision is the most acute human sense (Francis, 1977). Therefore, the overall quality of a food product can be assessed based on visual features alone. In the baking industry, light- but uniform-colored products are of a higher commercial value than dark-colored products. Taste panels are often carried out under colored light to prevent the effect of product color influencing the result. Correct color of foodstuffs assists in flavor perception and even identification (Martin, 1999). The nature of food processing tends to cause a loss of the original color of the raw foods. For this reason, colorants are sometime used to return food to a color that is perceived as attractive by the consumers. Color enhances the appearance of food, giving it esthetic value and an appetizing contrast. Dubose and Cardello (1980) discovered that as the intensity of the color increases, the perceived flavor increases accordingly. The research surmised that color influenced the anticipated oral and olfactory sensations because of the memory of previous eating experiences. In addition to color, size and shape are the other two commonly applied quality criteria for food grading. Most baked foods, such as crackers, muffins, bread, and other prepared snacks, have certain shape and size features that signify their overall quality. Thus, damage to these foods usually results in a change in object profile and shape. The dimensional characteristics of foods are also important because they allow food processors to determine how food will be handled during processing, and to understand why consumers prefer certain foods, as well as providing an indication of the product’s quality (Peleg, 1997). Control of thickness, diameter, and, to a lesser extent, weight is important, particularly in biscuit manufacture. In this industry, it was observed that large variability in the dimensions and weight of the food product causes production breakdown problems at the closely toleranced packaging stations, and can also result in excessively underweight and overweight packets that violate packaging legislation (Lawson, 1975; Cronin and Preis, 2000). Bakery products have many possible colors, shapes, and sizes. These attributes are influenced by many factors, including the ingredients used and the processing environment. Damage during handling and packaging adds more variety of color, shape,

Introduction 483

size, and other boundary irregularities. As far as the production of bakery products from frozen or fresh dough is concerned, consumers expect products with satisfactorily quality and sensory characteristics that should not differ much from the fresh ones. In particular, the development of color occurs more visibly during the later stage of baking; this attribute can also be used to judge the completion of the baking process (Wade, 1988). In addition to ingredient metering and dough-mixing, -freezing, -thawing, and -proving, baking constitutes one of the most important stages in the production of bakery items. The role of baking is to transform the predominantly fluid dough or batter into a predominantly solid baked product. Indirectly, baking alters the sensory properties of foods, improves palatability, and extends the range of tastes, aromas, and textures of foods produced from raw materials. Thus it is important to examine factors during baking which influence the quality characteristics of the final product. Baking triggers a series of physical, chemical, and biochemical reactions, resulting in changes in the bakery product, including volume expansion, evaporation of water, formation of a porous structure, denaturation of protein, gelatinization of starch, crust formation and the browning reaction, protein cross-linking, melting of fat and crystals and their incorporation into the surface of air cells, the rupture of gas cells, and sometimes fragmentation of cell walls (Sablani et al., 2002). At the macroscopic level, the most significant changes concern dimensions (length, width, and height), texture, water content, color, and flavor. The influence of product state parameters sensitive to these changes has been studied in various papers. According to Maache-Rezzoug et al. (1998), increases in length and reductions in the thickness and weight of baked outputs were strongly correlated with the concentrations of sugar, fat, water, and protein, and thus the flavor. Meanwhile, O’Brian and Morrisey (1989) reported that the excess of reducing sugars related to amino acids increased the non-enzymatic Maillard browning reaction, which is responsible for the formation of crust and darkening. Other studies reported that both intermediate and advanced Maillard reactions formed compounds that have pro-oxidant as well as mutagenic properties (Gazzani et al., 1987; Anese et al., 1999a). Results from these studies suggested that the antioxidant properties of baked foods could be reduced or enhanced depending on temperature, time, and moisture conditions. In general, an increase in the antioxidant potential is always associated with an increase in brownness; thus it can be hypothesized that compounds having pro-oxidant properties are formed in the early stages of the Maillard reaction. In order to produce a high-quality and consistent product, grading is an integral part of the baked goods industry. Traditionally, quality assurance methods used in the food industry have only involved human visual inspection. Such grading is carried out in two ways; by sensory and objective evaluation. In the first method, grading is performed principally using the human senses of touch and sight. Here, the products are inspected manually by trained human inspectors who make quality judgments based on “seeing” and “feeling.” All products sent to the packaging house are graded in this way, frequently without any objective standard on which the decision is based. Sampling is often a preferred choice, because hiring large numbers of adequately trained inspectors has become increasingly difficult. In this case, the inspectors draw several samples from the batch, inspect them for quality, and assign a grade to the entire batch based on the selected samples. Therefore, the decision is highly variable, and the process is


tedious, labor-intensive, and subjective. Moreover, it is extremely difficult to maintain consistency between panels and individuals, since human decisions vary from day to day and across inspectors. For this reason, sensory evaluations are often correlated and compared with objective physical measurements. Today, there is a wide assortment of sensors and transducers for the objective measurement of the size, shape, color, and weight of food materials. Giese (1995) provides an excellent review of sensors and equipment for measuring the physical properties of foodstuffs. Applications of these sensing technologies for the quality evaluation of foodstuffs, as baking indicators, and for control have widely been reported (Mabon, 1993; Pomeranz and Meloan, 1994; Lewicki, 2004). However, most of these technologies suffer drawbacks. First, their spatial resolution is often very limited. This is due to the fact that most data collected by such equipment are point-based. Consequently, the measurements do not represent the overall characteristics of the object in the image space. Secondly, the equipment is costly and difficult to maintain. As each characteristic requires a dedicated sensor, multiple sensors are needed to enable multiple measurements to be performed. This increases the cost, the quantity of equipment, and the complexity of the data processing. At present, machine vision sensors are also used to measure the physical properties of foods. The system performs measurements in two-dimensional space and produces images with improved spatial resolution. More importantly, machine vision sensors are programmable, implying that machines can be used to measure several food quality attributes. Machine vision systems are also useful for analyzing microscopic images, which is important for assessing internal product quality and studying the effects of variations in composition and processing parameters. They can also be used to automate process control, including guidance operations, and sorting, packing, and delivery systems.

2 Quality characteristics of bakery products The main ingredients of bakery products are flour, water, sugar, fat, and salt. A variety of shapes, colors, sizes, and textures may be produced by varying the proportions of these ingredients. Another factor which is of considerable importance in bakery is the dough. Being the intermediate product between the basic ingredients and the final product, the physical characteristics of the dough influence its machinability and hence the quality of the baked output. Properly baked dough then leads to a product with superior quality and sensory features. Fresh products usually present an appealing brownish and crunchy crust, a pleasant roasty aroma, fine slicing characteristics, a soft and elastic crumb texture, and a moist mouthfeel. A typical bakery process is illustrated in Figure 20.1 (Giannou et al., 2003). Referring to Figure 20.1, it can be seen that baking is the last and very important part of the process. This is the case regardless of whether the dough used is frozen or fresh. Baking induces many biochemical reactions within the dough, triggering a series of complex physicochemical changes ranging from the formation, development, deformation, and expansion of gas cells to modification of rheological properties. The most important physical changes are due to the Maillard browning reaction. This

Quality characteristics of bakery products 485

Raw material

Mixing

Molding

Packaging

Freezing

ThawingProofing

Mixing

Proofing

Baking

Figure 20.1 Bakery process showing the production of bakery products from frozen dough (dotted lines) and fresh dough (continuous lines) (Giannou et al., 2003).

process induces the development of the typical brown color and texture required in a good quality product. The most favored products have a uniform color, a smooth surface and shape, and a uniform granular and fine-pored texture. Information on color and other textural parameters is also useful in predicting changes in quality during baking, thereby enabling better process control and improvement of the appearance by optimizing the processing variables. Color is also measured as a guide to more practical matters that need to be considered in bakery processes. The treatment combinations with respect to the main ingredients, the processing conditions, and the amount of additives affect the Maillard reaction and hence color development during baking. The influence of product state parameter values on the browning reaction rate has been well studied in various papers. Some of the important discoveries are summarized in section 2.1.

2.1 Color The familiar color of baked product is the result of a complex system of biochemical reactions which are initiated when the surface temperature of the product exceeds 100◦ C. As stated previously, the so-called Maillard reaction is widely believed to be the main cause of darkening in food products. Browning reactions, which are some of the most important phenomena occurring in food during baking, represent an interesting research area regarding the implications for food stability and technology, as well as nutrition and health. According to Wade (1988), non-enzymatic browning is a two-stage kinetic reaction. In the first stage, highly reactive compounds such as the aldehydes, saturated ketones, unsaturated aldehydes and ketones, dicarbonyl derivatives, etc., are produced and accumulated. In the second stage, the polymerization of these compounds produces the brown polymers that are responsible for color formation. Of the compositional factors, the ratio between various forms of sugar and amino compounds has been repeatedly cited as the possible factor that determines the rate of darkening (O’Brien and Morrisey, 1989; Basier and Labuza, 1992). The incorporation of additives and other modifying agents capable of altering the rheological and physical characteristics of the product after cooking has long been established in the baking industry (Zadow and Hardham, 1981; Stahel, 1983). For instance, dairy proteins are frequently incorporated into the formulation of gluten-free breads since they offer both


nutritional and functional benefits, including flavor and texture enhancement, and storage improvement. Dairy products can also be used to increase water absorption, thereby enhancing the handling properties of the batter. The presence of dairy products in the formulation not only changes the textural and dietary properties of the breads, but also affects the baking characteristic of the crust and crumb. Gallagher et al. (2003) studied the effects of baking on color values and other quality parameters of gluten-free breads which were supplemented with seven dairy powders at four inclusion rates based on flour weight: 0, 3, 6, and 9 percent respectively. Altogether, seven different types of dairy products were investigated: molkin (mlk), demineralized whey powder (dwp), Kerrylac (klc), skim milk replacer (smr), skim milk powder (smp), sodium caseinate (nac) and milk protein isolate (mpi). Crust and crumb color were obtained through software in terms of CIELAB, with L∗ the lightness ranging from zero (black) to 100 (white), a∗ (redness) ranging from +60 (red) to −60 (green), and b∗ (yellowness) ranging from +60 (yellow) to −60 (blue). Each value was averaged by six measurements. Figure 20.2 summarizes the results. It can be seen from Figure 20.2a that the L∗ values of bread crust varied significantly from 62 to 36, corresponding to the 3 percent and 9 percent inclusion rates respectively. Also, breads containing dairy powders appear Crust 70

L*

60 50

0%

40

3%

30

6% 9%

20 10 0 mlk

dwp

klc

smr (a)

smp

nac

mpi

Crumb 5 4 L*/b*

0% 3

3% 6%

2

9% 1 0 mlk

dwp

klc

smr (b)

smp

nac

mpi

Figure 20.2 Effects of various types of dairy powders at different inclusion rates on (a) crust and (b) crumb color (Gallagher et al., 2003).


much darker compared to the gluten-free controls. Clearly in this case the presence of dairy powders influenced the Maillard browning reaction, and the browning rate increased with the increasing dairy powder content. However, the effect was generally small except in high protein-containing powders such as smr, nac, and mpi. The average L∗ value for other types of dairy powders was consistently less than 50 for all inclusion rates. As gluten-free breads tend to have a lighter crust color compared with white wheaten bread, the darkening of the crust resulting from the addition of dairy powders is highly desirable. On the other hand, the crumb’s color was influenced both by the powder type and by the level of addition. The average L∗ /b∗ values for mlk, klc, smr, and smp are consistently lower than for dwp, nac, and mpi, suggesting that the former group of dairy powders resulted in crumb-darkening while the latter group caused crumb-whitening. Since it is desirable to have a bread showing dark crust and white crumb characteristics, it can be concluded that the presence of protein-rich powders (such as smr, nac, and mpi) improved the quality of the product. In addition to the ingredients, another important factor which determines color quality is the cooking time. This factor has become critical in the past two decades with the invention of so called “high-temperature drying” technology. From the food manufacturer’s viewpoint, cooking carried out at high temperatures is a preferred choice because it leads to improvement in food color and firmness, lower baking loss, higher baking weight, and less stickiness (Dexter et al., 1981; Aktan and Khan, 1992). However, the surface browning of food cooked in hot air ovens can be uneven, an effect due to air velocity distribution. Consequently, different local heat fluxes are established during oven cooking. This means that the setting temperature can be reduced to achieve the same food quality in a hot air oven as in a static oven. Thus it is desirable to know the effect of cooking time and temperature on lightness variations and the degree of browning. According to Broyart et al. (1998), there are two stages during baking in which variation in lightness occurs; a lightening phase occurring during the early stages of baking, followed by a darkening phase. Previous observations seem to imply that the transition from the lightening to the darkening phase depends strongly on the product temperature. For biscuits, the maximal lightness corresponds to the time when the product temperature reaches a critical temperature located between 100◦ C and 110◦ C, at which point the browning reaction is initiated. At low air temperatures (usually less than 210◦ C), the product temperature does not reach critical value and thus no darkening happens. Modeling the lightness variation using a first-order differential equation and solving it using the Euler–Cauchy method, Broyart et al. (1998) were able to predict the effect of water content on darkening as a function of temperature. Figure 20.3 shows the results in a three-dimensional plot. It can be seen from Figure 20.3 that the darkening rate does not vary much with water content at low temperatures; in contrast, it exhibits a strong dependence on water content at higher temperatures. Furthermore, at a fixed product temperature and when the water content approaches 2–3 g per 100 g of dry matter, the darkening rate decreases considerably with the decreasing water content. Therefore, it can be inferred that a product with high moisture content at a high temperature darkens more quickly than does a drier product at the same temperature. It can also be seen from Figure 20.3 that for a fixed water content, the increased darkening rate with increasing product temperature exhibits two different

Darkening rate (min−1)


1 0.8 0.6 0.4 0.2 0 10 5

Water content (g/100 g of dry matter)

0 100

110

120

130

140

Product temperature (°C)

Figure 20.3 The effect of water content on the rate of darkening as a function of product temperature (Broyart et al., 1998).

patterns. At low moisture content the darkening rate changes vary slowly with product temperature, being almost negligibly small for a moisture content of less than 3 g per 100 g of dry matter. Beyond this value, and especially at a high moisture content, the darkening rate increases exponentially with product temperature. In other words, darkening will only become visible when the product temperature reaches a certain critical level. Broyart and colleagues performed several experiments in order to establish the relationship between product temperature and darkening, in which the product was baked at various temperatures – 180◦ C, 210◦ C, 240◦ C, 270◦ C, 300◦ C, and 330◦ C. At each temperature, the product lightness in terms of percentage L∗ values was monitored at fixed time intervals. These values were compared with the predicted values; Figure 20.4 summarizes the results. Both the simulation and experimental results in Figure 20.4 clearly indicate that the rate of change in product surface lightness is temperature dependent. Throughout the monitoring period, the lightness values remain relatively unchanged for temperatures below 240◦ C.Above this temperature, the L∗ (percent) started to drop after 2–4 minutes, indicating an increase in darkening after this cooking time. Figures 20.4d–20.4f show that the L∗ (percent) increased slightly at the initial stage of cooking, and then started to drop after 2–3 minutes. Therefore, color development during baking occurs sequentially in two phases: lightening and darkening. As expected, the rate of color change is more evident in the darkening phase. As the Maillard reaction has recently been associated with the formation of compounds with strong carcinogenic properties, it is important to know the relationship between color changes due to browning and the formation of compounds with antioxidant activity. Anese et al. (1999b) studied the effects of low, high, and very high temperatures on the development of color in pasta. The low-temperature drying process involved an oven temperature of 50◦ C and a cooking time of approximately 475 minutes. The high-temperature process was a multi-step procedure, in which 110◦ C was applied for 50 minutes followed by a lower temperature of 50◦ C for the next 125 ◦ minutes. The very high-temperature process involved maintaining 110 C for 220 min∗ b utes. By expressing Hunter color parameters as hue angle tan−1 a∗ , they performed

80

80

60

60

60

40

40

20

20

0

0 0

2 4 Time (min)

L*(%)

80 L*(%)

L*(%)


6

20 0 0

(a)

2 4 Time (min)

0

6

(b) 80

60

60

60 L* (%)

80

40

40 20

20 0

2 4 Time (min) (d)

6

6

40 20

0

0

2 4 Time (min) (c)

80 L* (%)

L* (%)

40

0 0

2 4 Time (min) (e)

6

0

2 4 Time (min) (f)

6

Figure 20.4 Relationship between product lightness and different air temperatures: (a) 180◦ C, (b) 210◦ C, (c) 240◦ C, (d) 270◦ C, (e) 300◦ C and (f) 330◦ C. The solid and dotted lines show measured and simulated results respectively (Broyart et al., 1998).

five measurements on each sample and the coefficients of variation, expressed as the percentage ratio between standard deviation and the mean value, were computed. Figure 20.5 summarizes the results. It can be observed from Figure 20.5 that no change of color occurred during low and very-high temperature cooking. In contrast, a significant change in hue angles was observed in the first stage of high temperature cooking. For both high and very-high temperature cooking, it can be seen that the antioxidant activity remained relatively low up to a moisture content of about 25 g/100 g, and steadily increased thereafter. Therefore, it can be concluded that the antioxidant or antiradical activity strongly correlates with temperature and moisture content. The results of these studies therefore indicate that the increase in antioxidant activity can be associated with the increase in brownness. The effect of cooking time on the browning property of other food materials has also been studied. Wang and Sun (2003) developed an algorithm based on computer vision for monitoring the color changes of cheese during heating. The cheeses used in their investigation were Cheddar and Mozzarella, because these two varieties are commonly used as toppings for pizza and other prepared foods. Image analysis was performed by capturing the cheese slice before and after cooking, extracting the gray values of the captured images, and calculating the cheese browning factor (BF), which is expressed as: BF =

GV o × 100% GV i

(20.1)


Antioxidant activity Hue angle

110

110

Hue angle (tg−1 b*/a*)

105

100 95 90 85

100 95 90 85

80

80 0

10

20

30

40

50

0

10

Moisture (g/100 g)

20

30

40

50

Moisture (g/100 g)

(a)

(b) 110


105 Hue angle (tg−1 b*/a*)

Hue angle (tg−1 b*/a*)

105


100 95 90 85 80 0

10

20

30

40

50

Moisture (g/100 g) (c) Figure 20.5 Changes in hue angle values and antioxidant activity of pasta samples subjected to (a) low temperature, (b) high temperature and (c) very high temperature cooking (Anese et al., 1999).

where GVo is the average gray value before cooking and GVi is the average ith gray value after cooking. In this case, GVi values were computed as a function of time and temperature. Figure 20.6 shows the results for heating Cheddar cheese. It can be seen from Figure 20.6a that the BF increased with increasing temperature, except at temperatures of 70◦ C and 100◦ where the BF increased sharply in the first 8 minutes of cooking. This trend continued for temperatures of 180◦ C and 200◦ C, but remained relatively constant for temperatures of 130◦ C and 160◦ C. Limited browning was observed at a temperature of 70◦ C. Meanwhile, Figure 20.6b shows that the BF increased when the cooking temperature exceeded 160◦ C, and was independent of cooking time. Similar results for surface browning were also observed for the Mozzarella cheese. These results suggest that browning is a temperature-induced reaction, occurring more profoundly at higher temperatures.

180

180

160

160 Browning factor (%)

Browning factor (%)


140

120

140

120 100

100

80 0

2

4

6

8

10 12 14 16 18 20 22 Time (min)

80 60

80

100

120 140 160 Temperature (°C)

70°C

100°C

130°C

1 min

2 min

160°C

180°C

200°C

10 min

12 min

Figure 20.6 Changes in browning factor of Cheddar cheese with increasing (a) cooking time and (b) temperature (Wang and Sun, 2003).

In addition to the cooking time, moisture content, and temperature, other factors affecting the quality appearance of bakery products include the incubation time and the ingredients used. Kara et al. (2005) studied the effects of high protease-activity flour (HPAF) on cookie quality parameters. They observed that both the lightness (L∗ ) and the yellowness (b∗ ) values decreased with increasing incubation time, while the redness (a∗ ) values increased. This means that the surface color of the cookies became darker with increasing incubation time. They deduced that the addition of protease into flour increased the release of small peptides and free amino acids, causing a higherlevel Maillard reaction. This was evident from the changes in both the L∗ and a∗ values with an increasing incubation period. Overall, their results indicate that HPAF altered the quality characteristics of the cookies, suggesting that this ingredient can be used as an economical alternative to other types of commercial proteases. In light of the above discussion, it can be concluded that color plays a very important role in quality control of bakery products. Color appearance can be used to judge the completion of baking process. Information on color is also useful in predicting changes in quality of cooking, thereby enabling better process control and improvement in appearance by optimizing the processing parameters. The color changes also appear to have a definite correlation with texture parameters – in general, an increase in darkening is accompanied by increasing rigidity (measured as firmness, stiffness or hardness) of most bakery products. Gao and Tan (1996) described texture in terms of cell size and density for the analysis of expanded food products. Their research entailed extensive development of color-image processing to characterize the cross-section and surface image texture of a test product. Performing correlation and regression analysis on image features and comparing them with SEM measurement, they deduced that a number of image features were highly descriptive of texture-related geometric

180 200 220

8 min


properties. This example indicates that the development of color occurs hand-in-hand with the development of structure seen in the hardening of most food products. It is also interesting to observe that a positive correlation between color changes or browning and the antioxidant activity of Maillard-reaction products exists in many food systems, such as pasta, cookie, bread and cereal products. Even though the mechanisms responsible for the formation of antioxidants are still not fully understood, research indicates that color can be considered to be an index of the overall antioxidant properties of foods. The ingredients used in the formulation also affect the overall color appearance of bakery product. Formulations with a high moisture gradient generally produce darker products compared with those with a reduced moisture gradient. This is particularly the case with cookie and biscuit recipes.

2.2 Rheological and textural properties As water, fat, and sugar constitute three essential ingredients used in making dough, rheological properties such as shape, size, and texture are very useful quality indicators of bakery products. A variety of shapes and textures may be produced by varying proportion of these ingredients. Of the three ingredients, water has a special role in dough formulation. Water determines the conformational state of biopolymers, affects the nature of interactions between various constituents of the formula, and contributes to dough structuring (Eliasson and Larsson, 1993). Bloskma and Bushuk (1988) observed that adding water to the formula reduces the viscosity and increases the extensibility of dough. If the proportion of water is too low, the dough becomes brittle, frequently exhibiting a marked crust effect due to rapid dehydration at the surface. Water leads to an increase in the consumption of total specific energy, a sharp decrease in dough viscosity, and a slight reduction in relaxation time. Consequently, the bakery products become longer and slightly less thick. Fat, on the other hand, contributes to the plasticity of the dough and acts as a lubricant. When present in large quantities, the lubricating effect of fat is so pronounced that very little water is needed to achieve a soft consistency. However, the effect of fat on the finished product depends strongly on the biochemical composition, particularly the protein content. According to De La Rocha and Fowler (1975), an increase in protein content tends to reduce the length of finished products after baking. The effect of sugar on the behavior of dough is well understood. Sugar makes the cooked product fragile, because it controls hydration and tends to disperse the protein and starch molecules, thereby preventing the formation of continuous mass (Bean and Setser, 1992). A more thorough study on the effects of water, fat, and sugar on rheological and textural properties of dough and on the quality of the final bakery products such as biscuits was also reported (Maache-Rezzoug et al., 1998). According to this study, the quantity of water significantly affected the behavior of the dough after baking. Figure 20.7 shows the effect of varying the quantity of water on the biscuit length, weight, and thickness. It can be observed from Figure 20.7a that the length increased most notably when the water concentration was greater than 21 percent. In contrast, both weight and thickness reduced slightly when the water concentration was increased. The slight reduction in weight is primarily due to tightening of the moisture distribution


58

8.0

4.5

7.5

4.2

Length Weight

7.0

55

Thickness (mm)

56 Weight (g)

Length (mm)

57

54

3.9

3.6

6.5 53 52 16

17

18

19 20 Water (%) (a)

21

3.3 16

6.0 23

22

17

18

19 20 Water (%) (b)

21

22

Figure 20.7 Relationship of the water concentration of the dough with (a) length and weight and (b) thickness of biscuit after baking (Maache-Rezzoug et al., 1998).

55 7.0 50 Weight Length

45 5

10

15 20 Fat (%) (a)

6.5

25

6.0 30

Weight (g)

Length (mm)

7.5

Thickness (mm)

4.3

60

40

4.5

8.0

65

4.1 3.9 3.7 3.5 3.3

5

10

15 20 Fat (%) (b)

25

30

Figure 20.8 Relationship of the fat concentration of the dough with (a) length and weight and (b) thickness of biscuit after baking (Maache-Rezzoug et al., 1998).

during baking, as wetter dough pieces lose proportionally more moisture than do dry pieces. The baking process attenuates the magnitude of weight variability by reducing the distribution of product moisture. Cronin and Preis (2000) have also reported that variations in the dimensions, weight, and thickness of commercial and laboratory Rich Tea type biscuits are sensitive to changes in the baking process, as well as the variability in dough moisture content. Figure 20.8a shows the effect of varying fat concentration on biscuit length, weight, and thickness. It can be seen that the presence of fat in the formula favored an increase in the product’s length, especially in the fat concentration range of 10–20 percent. Beyond this region, the length tended to stabilize. The increase was approximately 20 percent, which was significantly larger than the increase in length resulting from an increase of water concentration. A quasi-linear decrease in weight and thickness was

23


Length (mm)

60

8.5

4.6

8.0

4.4

55 7.5 50 45 40

25

30

35 40 50 Sugar (%) (a)

60

Thickness (mm)

Weight Length

Weight (g)

65

4.2

7.0

4.0

6.5

3.8

25

30

35 40 Sugar (%) (b)

50

60

Figure 20.9 Relationship of the sugar concentration of the dough with (a) length and weight and (b) thickness of biscuit after baking (Maache-Rezzoug et al., 1998).

also observed when the fat content increased. The decrease in thickness seemed to be more significant for fat concentrations of 10–20 percent. Figure 20.9 illustrates the effect of sugar on biscuit length, weight, and thickness. It is evident that the sugar concentration showed a significantly positive correlation with length but a negative correlation with weight. In fact, both length and weight varied in an almost linear fashion with the increase in sugar concentration. The drop in weight was probably due to slight shrinkage of the dough. It can also be observed that the thickness decreased considerably when the sugar content was increased – therefore, the higher the sugar concentration, the thinner the finished product. Sugar also influenced the mechanical properties of the dough: an increase in sugar concentration resulted in an increase in specific tearing force, indicating greater crispness of the product (MaacheRezzoug et al., 1998). This finding is in agreement with the quality characteristics of baked tortilla chips as studied by Kayacier and Singh (2003). Their research has shown that the texture of tortilla chips depends on several factors, including raw materials and baking conditions. Textural attributes of chips baked at temperatures higher than 232◦ C followed a polynomial function with time, increasing until a certain time was reached and then decreasing. These changes were attributed to the formation of air cells and cracks in the structure of chips. The effect was more pronounced at higher temperatures, indicating that a shorter time was required for samples to reach similar textural attributes when baked at a higher temperature. Results from tortilla chips studies indicated that when determining the optimum baking conditions for desired textural properties, the formation of structure should be considered. In summary, the length or spread of biscuits, which is one of the more important quality parameters, is positively and significantly correlated with water, fat, and sugar contents of the dough. Of these ingredients, sugar content shows the greatest correlation with spread. It has also been reported that another rheological characteristic which has shown great correlation with spread is the elastic recovery, which indirectly provides information on product hardness, stickiness, and adhesiveness (Manohar and Haridas, 2002). This research showed that both the spread and the thickness of biscuits were found to be significantly correlated with surface characteristics as well as the texture

Computer vision inspection of bakery products 495

of biscuits. Therefore, it can be assumed that the spread ratio or length can be used as one predictor of biscuit quality.

3 Computer vision inspection of bakery products It is clear from the above discussion that the physical properties of baked products, such as color, size, and shape, are important quality attributes of most baked foods. Large variability in these attributes can potentially cause production loss if there are machine breakdowns and wastage of energy, but, more significantly, such variability has an influence on consumer perception of quality and thus determines the level of acceptability prior to purchase. For most bakery products, the physical variation between individual items of the same type exists due to the different factors discussed above. This variation is one of the main challenges for the industry, as the increased purchasing power and awareness of food quality among consumers has resulted in the demand for products of high uniform quality. Therefore, it is in the interest of the manufacturer to ensure that goods leaving the plant are of as high a standard as possible, including cosmetically. Manual inspection of large numbers of items on the processing line is very expensive as well as being unreliable due to the workers’ finite attention span and limited visual acuity. Non-visual inspection, such as touching the edge of the product, may damage delicate foodstuffs as well as introducing bacteria. The need for visual quality control, along with the increasing scrutiny of hygiene standards and factory automation, leads to a demand for automatic and objective evaluation of visual parameters such as color, size, and shape. Machine vision systems in the form of cameras, illumination, frame grabbers, and computers provide a solution that may satisfy this demand.

3.1 Color inspection As stated above, most baked products consist basically of sugar, eggs, flour, baking powder and water. Once these ingredients have been combined and the mixture appropriately shaped, it is baked in a hot oven for a specific time. When displayed, the baked product exhibits an attractive, shining, golden-brown surface, normally referred to as the “top part.” As the baking time is rather short and the temperature inside the oven is not uniformly distributed, variation in color is likely to happen. This variation indicates the degree of cooking and thus the product quality. Muffins provide an example of a bakery product that exhibits these properties. Figure 20.10 shows a group of four blueberry muffin images serving as references for different degrees of “doneness” from a single batch (McConnel and Blau, 1995). Theoretically, there are many more groups with different levels of doneness that can be used to categorize muffins, but only four categories are shown and discussed here. Careful examination of Figure 20.10 reveals that muffin images are multimodal, exhibiting at least four different clusters. The first cluster consists of the light-brown regions belonging to the cake itself. The second cluster is the blueberries, and exhibits darker regions. The foil or the background, showing as gray and white regions,


(a)

(b)

(c)

(d)

Figure 20.10 Group of muffin images captured from a single batch after baking: (a) undercooked; (b) optimally cooked; (c) slightly over-cooked; (d) substantially over-cooked (McConnel and Blau, 1995).

constitutes the third cluster. Finally, the area occupying the region between the muffin and the foil belongs to the shadow cluster. Essentially, the shadow and background clusters remain relatively constant, providing no information regarding the degree of doneness of the muffin; only the cake and blueberry clusters are useful for characterizing the degree of doneness. The challenge for the image-processing software is therefore to use this information as the basis for color classification. Classifying an object with pure color density is very straightforward, since conventional methods such as thresholding, look-up table, and nearest-neighbor methods work quite well. While objects that are relatively simple in texture may be separated into their subcomponents using gray-scale image analysis, more complicated objects like the muffin images shown in Figure 20.10, with varying color composition, do not readily lend themselves to segmentation in gray scale. Therefore, it is necessary to involve the color characteristics of the objects in order to obtain a proper classification and to perform subsequent analysis. Chapter 3 provides the theoretical background regarding machine perception of color, and discusses different types of color models which can be used to specify colors digitally. Two of the most popular models used in computer vision applications are the software-oriented RGB (red, green, blue) and hardware-based HSI (hue, saturation, and intensity) models (Chen et al., 1998). The following paragraphs describe how these color models have been successfully used in machine vision inspection of baked goods. In the study of color development of commercial biscuit product samples, Hamey et al. (1998) used the RGB color space to monitor color changes throughout the baking process. Biscuit images were captured using a sky-blue background in order to facilitate image segmentation and calibration. Each pixel of the camera’s R, G, and B signals was first digitized and then projected into three-dimensional color space by linear transformation. Figure 20.11a shows the typical biscuit images, while Figure 20.11b displays the color distribution of this food sample plotted in RGB color space.


White

Green

Zone 1 Zone 2 Raw

Blue Zone 3 Red Charcoal Black (a)

(b)

Figure 20.11 Images showing group of (a) milk-coffee biscuit samples and (b) the resulting color distribution plotted at different baking time (Hamey et al., 1998).

Referring to Figure 20.11b, it can be observed that the color development of a biscuit follows a curve which uniquely characterizes the entire baking process, from raw dough to overbaking. This curve is known as the baking curve. Since the biscuit color changes along the baking curve, Hamey et al. (1998) categorized each biscuit into three distinct regions – zones 1, 2, and 3 – corresponding to undercooked, correctly cooked, and overcooked, respectively. The feed-forward neural network (FFNN) with backpropagation (BP) learning algorithm was trained to recognize the biscuit and categorize it into one of the above groups. Since color is represented in RGB space, some form of pre-processing is needed in order to reduce the input dimension and hence overcome the speed limitations associated with color-based machine inspection. The self-organizing map (SOM) was employed to achieve this. In this case the SOM used was a Kahonentype neural network, which is characterized by its ability to create the topological feature map that models the probability distribution of the training samples (Hiotis, 1993). The trained SOM produces N × 1 output vectors or nodes, and a baking curve is formed when these nodes are connected. The curve is then projected into the RGB space, from which a histogram depicting the color distribution of the biscuit sample is constructed. Determining the SOM’s output nodes is paramount in ensuring that the baking curve is preserved and all essential features related to biscuit color development are captured. In this study, it was discovered that the optimal SOM comprised 20 output nodes, constructed using 298 biscuit samples and after 20 iterative cycles. The overall strategy for biscuit classification is summarized in Figure 20.12. In summary, their results indicate that the proposed system is at least 41 percent better than a human inspector, with the 8-2-1 FFNN being the optimal architecture for this application.


Start Color image Segmentation Segmented image Extract RGB pixels RGB pixels Baking category

SOM Histogram of baking curve

Grade 1: under-bake Grade 2: correct-bake Grade 3: over-bake

Trained FFNN?

No

Train FFNN

Yes Trained FFNN

Grade biscuit

Stop Figure 20.12 The overall strategy for biscuit inspection system employing SOM for extracting the baking curve and FFNN for color classification. The dotted lines indicate one extra step needed in testing (Hamey et al., 1998).

Another baked food in which surface color plays a significant role in determining the product’s quality is the muffin. Strategically, there is great potential in applying the vision system for muffin inspection, since muffins are being produced in their thousands every day, in a variety of colors and shapes. An automated vision system would help in standardizing quality evaluation by eliminating human subjectivity and inconsistency. Abdullah et al. (2000, 2002) investigated the use of computer vision technology to grade a brown muffin based on machine learning of color. One of the ultimate objectives of this research was to use digital images for automated inspection of muffins as well as other baked foods. They carried out visual inspection of muffins by instrumental assessment of the top surface color using a Minolta colorimeter meter that can spectrally approximate eye function in terms of L∗ , a∗ , and b∗ values. It was discovered that the desirable color of muffins, as indicated by L∗ , a∗ , and b∗ values, falls in the ranges 42.27 to 47.55, −8.67 to −6.93, and 17.27 to 19.17, respectively. Based on these measurements, they then categorized the muffins into three distinct groups reflecting three different degrees of doneness: undercooked, moderately cooked, and overcooked. The average L∗ , a∗ , and b∗ values, respectively, were 47.55, −6.93, and 19.17 for the


(a)

(b)

(c)

Figure 20.13 Digital images of brown muffin showing variation of colors corresponding to three degree of doneness: (a) undercooked; (b) moderately cooked; (c) overcooked (Abdullah et al., 2002).

undercooked group; 46.50, −7.55, and 18.13 for the moderately cooked group; and 42.42, 8.67, and 17.27 for the overcooked group. Figure 20.13 shows the images of muffin samples. As stated previously, the muffin image, like most baked-product images, is multimodal, with several distinct color clusters. From the quality viewpoint, one of the clusters of interest is the cake cluster. This cluster varies in quality, shifting from one color space to another, indicating a strong relationship between surface color and the property of interest, or “doneness,” but the number of pixels remains relatively constant in quantity. In contrast, the number of pixels in the shadow and the background remains relatively constant in quality, but varies in quantity. The shadow and background clusters are therefore not useful for muffin grading. Clearly, any method that performs color classification must address this problem and be sensitive to changes in the varying components. The muffin images shown in Figure 20.13 were obtained using an 8-bit TMC-RGB ½ CCD color camera equipped with a 20-mm optical lens of the C-mount type, manufactured by PULNiX America Corporation, Sunnyvale, California. The camera was installed inside the conical chamber and illuminated with a cool-type ultra highfrequency fluorescent light ring. The fluorescent bulb was fitted with a continuous light intensity control which allowed 10 percent to 100 percent intensity adjustment. These arrangements produced ripple-free and almost 360◦ diffuse illumination, which was important for accurate machine vision measurement. In analyzing color, the HSI space was selected not only because of its close representation of human perception of color, but also, and more importantly, because it helped to compress information for easier color discrimination and a manageable solution. In this space only the hue component (H ) was analyzed, since this attribute directly characterizes the color properties of an object and thus the degree of doneness of the muffin. Since the vision system sees color in RGB space, transformation of RGB data to HSI color space is needed. On an 8-bit machine vision system, such a transformation is given mathematically by (Gonzalez and Wintz, 1987): ⎧ ⎤⎫ ⎡ ⎪ ⎪ ⎨ ⎬ 255 ◦ −1 ⎢ −0.5[(R − G) + (R + B)] ⎥ h = 360 − cos ⎣

for B ≥ G ⎦ × ⎪ ⎪ 360 ⎩ (R − G)2 + (R − B)(G − B) ⎭ (20.2)


1600 Under-cooked Moderately cooked over-cooked

1400

Pixel count

1200 1000 800 600 400 200 0

0 (Red)

10 (Dark-brown)

20 30 (Light-brown)

40

50

60 (Yellow)

Hue Figure 20.14 Histogram plot showing hue distributions corresponding to muffin images in Figure 20.13. These distributions are plotted by averaging over 10 samples for each group category (Abdullah et al., 2002).

or ⎤⎫ ⎪ ⎬ 255 −1 ⎢ −0.5[(R − G) + (R + B)] ⎥ h = cos ⎣

⎦ × ⎪ 360 ⎪ ⎩ (R − G)2 + (R − B)(G − B) ⎭ ⎧ ⎪ ⎨

⎡

for

B 0.89). Additional research by Symons and Dexter (1991) demonstrated that analysis of flour sample images, captured with a RGB color camera when exposed to a 365-nm excitation wavelength light, using in-house developed software, revealed that pericarp tissue was strongly correlated (r 2 > 0.9) with tristimulus color coordinate L∗ (brightness) of the flour, ash content, and flour grade color.

530 Image Analysis of Oriental Noodles

As noodle appearance and color are the consumer’s initial quality parameters, there is a preference for manufacturers to use high-quality, low-yield patent flours, as they result in brighter noodles (independent of type) and display fewer areas of discoloration or speckiness. Rapid changes in noodle appearance occur over time, with the greatest change being observed within the first few hours after production and slowly declining thereafter. Darkening of the noodle sheet occurs over the entire dough sheet matrix, and is accelerated in small areas. The reason for the accelerated localization of color instability or change is believed to be the loci of contaminating bran particles. As previously discussed, the bran material is an abundant source of not only enzymes but also their substrates. The common procedure for measuring noodle color is to use a colorimeter and measure the CIE 1976 brightness (L∗ ), redness (a∗ ), and yellowness (b*). However, this technique does not allow the manufacturer the ability to determine the degree of noodle speckiness, which quantifies the color differences of these specks from the background matrix, or to assign relative contributions of specks to the overall color components. Image analysis of noodles offers the manufacturer the ability to address these concerns, as well as explore the impact of various factors or formulations on noodle appearance. An added benefit that is becoming increasingly important in today’s international marketplace is the ability to standardize methods across different production facilities, to enforce the identical degree of quality control and to retain both records and images of the product. The use of defined algorithms to quantify various components is essential to those manufacturers seeking ISO 1702-5 accreditation.

2.2 Initial noodle-imaging research Initial work (Hatcher et al., 1999) demonstrated that image analysis could be effectively used to quantify, measure, and discriminate varietal differences in white seed-coated wheat varieties in the preparation of yellow alkaline noodles prepared from high-quality patent flours. Noodle images were captured using a color camera (CD-950, Sony Canada) attached to a macroscope (Model M-8, Wild Leitz, Canada). The imaging of the noodles required additional lighting supplied by a fiber optic 3200-K ring lamp, with the images being captured via a frame-grabber and software. Illumination consistency was achieved through the use of a Kodak No. 3 gray scale (Eastman-Kodak, NY), and by adjusting the camera gain. Images of the gray scale and a standardized white tile were captured at the beginning of each daily experiment. Their preliminary experimentation highlighted setting the No. 3 gray-scale image to 140 on the 0–255 scale to yield optimum discrimination when the image of the noodle surface was corrected for any variations. In order to quantify noodle speckiness, they utilized a minimum difference in darkness between discolored specks and the surface of the noodle, referred to as the delta () gray, and examined noodles at -gray values of 2, 5, and 10. The second parameter employed to discriminate specks in their 1999 study was a minimum threshold speck size of 5, 10, or 15 pixels, corresponding to actual sizes of 3, 6.3, or 9.4 × 10−3 mm2 (Figure 21.6). In their research the noodle sheet was positioned under the macroscope/camera system using a fixed grid positioning system, which allowed for repetitive images to be consistently captured. Images (six) from

Imaging in noodle quality assessment 531

0 2 5 10

255

Figure 21.6 Setting gray will influence the subsequent size of the speck detected, based upon individual pixels, and determine whether it meets the minimum threshold size for detection.

different positions on the noodle sheet were captured, with each image representing a 1.5 × 1.1-cm section of the noodle, or 9.9 cm2 of the noodle sheet. A drawback for image analyses during this first investigation was the significant variation in noodle surface color due to uneven hydration within the first hour of production. However, by 2 h post-production the algorithms employed to address the uneven hydration were found to have overcome this issue. They observed an increase in the number of specks detected over a 24-h period using this technique, which was able to show significant differences between the number of specks detected due to wheat variety. Utilizing three separate noodle sheets produced from high-quality patent flours on different days, with a -gray value = 2, 62 ± 3.45 specks were detected in one variety while only 45 ± 2.5 were detected in the other at 24 h post-production. It was noted that while images had been captured during the 2–8-h post-production period, there were no statistical differences until the 24-h reading. They also reported that the number of specks detected decreased with increasing -gray values, although no linear relationship was observed. It was not unexpected that maximum variation occurred at a -gray setting of 2. A significant linear relationship was detected for both -gray settings of 5 and 10 between the number of specks and the aging period (r 2 = 0.75–0.99 for one variety, while the less specky variety displayed r 2 = 0.26–0.89). Investigation of the role of minimum threshold speck size yielded the anticipated decrease in the number of specks with increasing threshold speck size. However, while the numbers decreased, changing the minimum threshold speck size did not alter the linear relationships between the number of specks detected and time. This technique also allowed the investigators to examine whether the size of the specks detected under the combination of parameters varied over time. While the influence of the minimum threshold size for detection was very significant (P = 0.0001), no change in the mean speck size was observed between either variety over time, suggesting that the bran itself was the key and that there were no additional condensation reactions expanding the speck size past the perimeter of the bran speck itself. This leant credence to the concept that at the production absorption level of 32 percent there is insufficient free water available for the enzymes, substrates or subsequent condensation products to migrate.


One of the advantages image-analysis systems bring to the researcher, which is not present with a standard colorimeter, is the ability to characterize the degree of darkness contributed by individual specks and their calculated mean darkness. Hatcher and co-workers (1999) demonstrated that the largest change in speck-darkening occurred initially, followed by a decreasingly slower change over the 8-h post-production period. Significant differences in the mean darkness intensity of the specks between the two white seed-coat varieties were detected as early as 3 h after production, regardless of the -gray or minimum threshold setting. This technique also allows the researcher the capability to quantify and analyze the relative distribution of speck discoloration over time. In the preliminary work of Hatcher et al. (1999), it was demonstrated that by as early as 2 h post-production a highly significant difference in the distribution of speck darkness between the two white seed-coat varieties and aging of the noodle sheets was emphasized. The more desirable variety yielded 36.8 percent of its specks with brightness of 140, while the other variety was unable to achieve this goal and was subjectively assessed visually by the researchers to be less desirable. This difference was also detected instrumentally by a colorimeter, where the noodles’ redness (a*) at 24 h was significantly different, i.e. 0.22 versus 0.62. The use of image analysis of noodles to detect differences in the mill’s degree of flour refinement was conducted by Hatcher and Symons (2000a) for both yellow alkaline and white salted noodles. Two similar white wheat cultivars were milled to yield both patent (60 percent yield) and straight-grade flours, and noodles were prepared and analyzed (Hatcher et al., 1999). In this work they demonstrated that while no significant difference could be detected between the varieties’ patent alkaline noodles at 2 h postproduction (12.4 vs 15.5 specks), noodles prepared from straight-grade flour displayed significantly higher numbers of specks than did their patent counterparts (31.9 and 28.3 specks/image). While the aging alkaline noodles displayed the same phenomena as previously observed (Hatcher et al., 1999), with varietal distinction detected in the number of specks at 24 h in patent noodles, straight-grade flour noodles showed a significantly greater number of specks (Hatcher et al., 1999). In one variety the straight-grade alkaline noodles yielded a greater than 50 percent increase in specks relative to the corresponding patent noodles. This research was also the first to directly compare alkaline and white salted noodles, prepared from the same flours, by image analysis. The most dominant distinction between white salted and alkaline noodles via image analysis was the lower number of discolored specks detected in either flour or variety, relative to their alkaline counterparts. At the initial 2-h post-production period, the white salted patent noodles displayed less than half the number of specks of the alkaline noodles, while noodles prepared from the straight-grade flour had less than 40 percent of the corresponding number of alkaline-noodle detectable specks. During the time-course measurement period of 1–24 h, the white salted noodles displayed no more than 59.6 percent of the number of specks observed in either variety or flour-type alkaline noodles, which confirmed that speck development was directly related to the type of noodle being produced. Another interesting development of this research was that while white salted straight-grade noodles displayed more specks than did those from patent flours, flour refinement did not


significantly influence the number of specks, which was in contrast to observations with alkaline noodles. Consistent with their previous research (Hatcher et al., 1999), at maximum sensitivity no cultivar or noodle-type effect was detected on speck size at 2 h for the noodles prepared from patent flours. However, significant varietal differences in speck size were observed in both white salted and yellow alkaline noodles in noodles prepared using straight-grade flours, with the variety Karma displaying larger specks than Vista. These differences, however, were not attributable to the type of noodle produced. Human visual (sensory) assessment is subjective, and it has been demonstrated that the size and darkness of a speck are exaggerated depending on the degree of contrast with the background (Francis and Clydesdale, 1975; Hutchings, 1994). While image analyses objectively demonstrated that the degree of noodle-speck darkness was not different between the two white wheat varieties’ alkaline noodles at 2 h, it surprisingly revealed that the alkaline noodle specks were lighter than those of the corresponding white salted noodles. Furthermore, differences in speck darkness due to wheat variety were detectable in the patent white salted noodles (Hatcher and Symons, 2000a). Flour refinement was shown to influence speck darkness because, in either white salted or yellow alkaline noodles, straight-grade flour yielded darker specks than the patent flours for both varieties. Changing the -gray parameters from 2 to 5 or 10 had a minimal discernible impact on speck darkness. While mean speck darkness was not influenced by noodle type (i.e. 141.9 for variety A alkaline versus 138.3 for white salted) at 2 h, very distinct distributional differences were observed. Examination of their respective speck distribution profiles revealed that 94.6 percent of alkaline noodles and 87.3 percent of the salted noodle specks were above the 130 value, which would account for the inability to distinguish the noodlesheet specks on the basis of mean darkness. However, further analyses of the alkaline noodle specks indicated that 21.4 percent were extremely light, falling in the 150–159 range, and 43.8 percent fell in the 140–149 range. The corresponding white salted noodle specks had 68.7 percent in the 130–139 range, with less than 10 percent in either of the other two lighter ranges. This distribution would distort the consumer perception of the appearance of the noodles, causing them to rate the specks as darker and larger in the white salted noodle due to the contrast perception problem. Further investigation revealed that for other varietal differences, one variety, i.e. the straight grade flour (either alkaline or salted), had specks exhibiting lightness values above 130, whereas the other variety had almost 40 percent of its specks above this value. This fact highlighted a discernible variety influence of which noodle manufacturers must be aware. Aging the various noodles until 24 h exaggerated differences due to variety, flour refinement, and type of noodle. Until recently, the majority of Asian noodle manufacturers preferred noodle products to be derived from white seed-coated wheat as compared to red seed-coat material. Initial research had used image analysis to characterize the white wheat products, but it was not until 2000 that red seed-coat material was investigated (Figure 21.7) (Hatcher and Symons, 2000b). Examination of patent and straight-grade flour alkaline noodles derived from red seed-coat wheat indicated that the number of detectable specks increased over time, reaching a maximum at 7 h post-production and declining


250

CWRS CWHWS

Specks/Image

200 150 100 50 0 F

M1

M2 M3

Q2

B12 M4

B3

M5

B4

TL LG

Millstream 2

Figure 21.7 The number of specks/25 cm of noodle sheet, at 24 hours post-production, prepared from different mill streams of Canada Western Red Spring or Canada Western Hard White Spring wheat.

slightly thereafter. The reason for this decline in the 7–24-h period (which was not observed in the white seed-coat material) was that the general noodle background matrix had darkened as well, thereby reducing the ability of the algorithms to detect specks. As observed previously, the noodles prepared from the patent flour displayed significantly fewer specks than did noodles prepared from the straight-grade material (34.8 vs 53.1) at 1 h after production. The influence of the type of noodle (salted versus alkaline) on red seed-coat derived flour displayed the same trend as observed in the white wheat material, with significantly fewer specks observed compared with the alkaline. Flour refinement effects also remained consistent, with the straight-grade noodles having slightly elevated speck numbers compared to their patent counterparts at each time interval. However, for white salted noodles prepared from Canada Western Red Spring (CWRS) wheat, neither flour displayed a significant increase in speck numbers over time. In both flours, the number of specks detected in the noodles at 1 h post-production remained relatively constant over the 24-h examination period. Furthermore, the number of specks detected in either patent or straight-grade CWRS white salted noodles was equivalent to that observed in the white wheats (Hatcher and Symons, 2000b). It was noted that changing the sensitivity, gray, or threshold limit did influence the number of specks detected, but did not alter the trend. Noodles prepared from the CWRS wheat displayed a greater loss of brightness (L∗ ) over time as compared to the white wheat when assessed by a colorimeter. Examination of speck darkness in CWRS noodles indicated that the same trend (no significant difference in mean speck darkness) observed in white seed-coated wheat was detectable between white salted (135.3) and alkaline (134.0) at 1 h after production in patent flour noodles. A similar observation was noted for specks in the straight-grade noodles, although they were darker than their patent counterparts. However, aging for just one additional hour did highlight differences between the red and white seed-coated wheat noodles. The red CWRS alkaline noodles aged for 2 h displayed a mean speck darkness of 127.3, which was significantly darker than those of the white wheat varieties previously investigated at the same time period (141.9 and 138.1). Examination of the corresponding CWRS patent white salted noodles indicated that noodle type was an influencing factor. While the CWRS mean speck darkness of 128.6 was not


significantly different from the white wheat variety Karma (125.8) at the same 2-h period, it was significantly darker than the white variety Vista (138.3). Surprisingly, this feature did not manifest itself in white salted noodles prepared from straight-grade flours, and thus demonstrated that there was no clear evidence to suggest that noodle specks from red-coated grains were always darker than those derived from white-coated material. The explanation for the difference due to seed-coat color may originate in the fact that it is common practice for noodles which are not sold within the first day after production to be incorporated judiciously in the following day’s material. Storage of the CWRS raw noodles for 24 h caused very distinctive darkening of the specks – considerably darker than the previously studied white wheat materials – in both the patent (106.6) and straight-grade (84.6) alkaline noodles. However, the influence of variety and noodle type was found to be important in white salted noodle-speck characterization. While the CWRS showed distinctly darker spots than the white wheat variety Vista after 24 h, it was not different from the white variety Karma. Examination of CWRS mean speck darkness was unable to discern any differences in speck darkness due to noodle type in patent flour noodles at the second hour. However, they did display vastly different speck-darkness distribution profiles. CWRS white salted noodle specks yielded 44 percent of their spots above the 130 gray level, while the alkaline noodles achieved only 28.7 percent. This difference in distribution profiles would clearly cause the consumer to perceive a difference. Furthermore, being compared to the white seed-coated varieties in previous research (Hatcher and Symons, 2000a, 2000b), a major distributional shift was highlighted, as the white seed-coated line Vista displayed 94.6 percent of its alkaline specks above 130 and 87.3 percent of its salted noodles above this level at the corresponding time period. Also importantly, examination of either type of the CWRS straight-grade noodles at 2 h had any spots lighter than 130 and only less than 3 percent in the 120–129 grouping. Aging the noodles for 24 h exaggerated the differences in the speck-distribution profiles, highlighting both the degree of flour refinement and the type of noodle. CWRS patent-flour noodles of either type, at 24 h, had a significant portion of their specks above the 110 gray level, yet the corresponding noodles prepared from their straight-grade flour had no material this light. Contrasting these findings, the corresponding white wheat noodles from the variety Vista yielded 36.9 percent of their specks above the 110 level, while the variety Karma continued to resemble the CWRS samples. Initial research (Hatcher et al., 1999) had been able to discriminate between two different registered varieties of CPSW derived from commercial shipments of grain. These individual shipments, while unique in terms of variety, represented a blend of growing locations within 1 year. Recently, research (Ambalamaatil et al., 2002; Hatcher et al., 2006) has unveiled the ability of image analysis of noodles to provide insight into the genotype, environment, and genotype × environment interactions as they pertain to noodle appearance (Figure 21.8). Within a 3-year study, during which flour mill yield remained constant, differences due to growing location, variety, as well as year, were found significantly to influence the number of specks detected within two classes of Canadian wheat. Further analysis also highlighted the very significance (P < 0.0001) of these three variables on speck darkness, as well as their respective interaction terms.


500

Specks/25 cm2

400 300 200 100 0 Domain CWRS Red coat

Barrie

Kernen

Kanata

Swift current

Snowbird CWHWS White coat

BW275

I.Head

Figure 21.8 The influence of different seed-coat color on the number of noodle specks detected at 24 hours after production from wheat grown using two different Canadian classes, Canada Western Red Spring (two varieties) and Canada Western Hard White Spring wheat (three cultivars), grown at three sites in western Canada.

3 Measuring the impact of external grading factors 3.1 Sprout damage Image analysis of noodles can also play a significant role in understanding and quantifying the impact of quality parameters of noodles. Sprout damage, as assessed by the Hagberg Falling Number test, is a common quality specification employed by the international market. Premature sprouting of wheat due to excessive moisture results in the production of undesirable levels of the enzyme alpha amylase. This enzyme can increase over 50 000-fold and have very deleterious effects on bread making, although the impact on noodle quality is not as pronounced. Investigation of severely sprouted wheat showed that alkaline noodles had the greatest number of specks/image at 1 h, and this was doubled by 7 h post-production. These numbers represented an approximate five-fold increase over corresponding sound, non-sprouted, identical control wheat (Hatcher and Symons, 2000c). As observed in previous studies, white salted noodles made with the same flours yielded significantly fewer specks at both time intervals. This study demonstrated that there was a significant difference in the size of the discolored specks over time due to the sprout damage, with the largest specks being detected in alkaline noodles prepared from heavily sprouted wheat flour. Noodles which had been allowed to germinate for 5 days displayed an almost three-fold increase in speck size, while white salted yielded a two-fold increase. It is thought that in the severely sprouted grain there is also a preponderance of proteolytic and other oxidative enzymes, such as peroxidase and polyphenol oxidase. Auto-oxidation of phenolics, particularly in the alkaline environment, in combination with degraded proteins, causes the differences in sizes between the alkaline and white salted noodles (Hatcher and Symons, 2000c).

Measuring the impact of external grading factors 537

An additional problem noted with the use of sprouted flour was that for both types of noodles the mean darkness of the specks was significantly darker than in their nonsprouted controls. Further analysis of the darkness distribution profiles demonstrated differences between the type of noodle and the impact of sprout damage. As early as 1 h post-production, alkaline noodles revealed differences due to sprout damage, compared with their control. The sprout-free (control) noodles had 81.6 percent of their specks falling above the 110 gray level, yet noodles prepared from 3-day germinated material had only 21.4 percent in this lighter range. Noodles made from severely sprouted material (5 days germinated) had no specks above this level, and all of them were below the value of 100 at the earliest stage of production. Measurements conducted 7 h after preparation indicated that while the sound (control) flour still retained 73.5 percent of its specks above the 110 level, none of the 3-day germinated material achieved this value. The majority of specks (87.8 percent) of the severely sprouted flour noodles were significantly darker, ranging between the 50 and 59 gray-level range. Assessing the severely sprouted material after 24 h highlighted their further darkening, as 94 percent fell below the gray level value of 50. In contrast, white salted noodles prepared from sound flour had 98 percent of their specks with the gray value above 110 at 1 h. The same phenomenon was found in the 3-day germinated material. The severely sprouted (5-day germinated) also retained the majority of specks (57 percent) above this value. Aging for 24 h demonstrated that while the severely sprouted flour alkaline noodles had 94 percent of their specks below the gray value of 50, the corresponding white salted noodles had only 0.2 percent in the same range. Polyphenol oxidase is responsible for the discoloration of fruits and vegetables (Mayer and Harel, 1991; Zawistoski et al., 1991; Osuga et al., 1994). The use of sulfite either as a wash or to be directly incorporated into the product has been a means of constraining food discoloration (Lambrecht, 1995). Incorporation of sodium metabisulfite into flours with varying degrees of sprout damage caused a significant decline in the number of specks/image detected in the alkaline noodles, even in the severely sprouted material. This influence was also observed in the non-sprouted material over the 24-h period, as only 18 specks were detected with the inhibitor present, but 78 without.

3.2 Fusarium head blight Fusarium head blight (FHB), also referred to as scab, can cause significant loss of wheat yield, and has additional health implications due to the potential production of mycotoxins – the most notable being deoxynivalenol (DON). Alkaline noodles prepared from either patent or straight-grade flours derived from Canada Western Red Winter (CWRW) wheat samples (eight) with fusarium damage levels ranging from 0.5 to 9.6 percent, which encompasses the top No.1 grade to feed, were investigated (Hatcher et al., 2003). The commercial samples were < 90 percent one specific variety, CDC (Crop Development Center) Kestrel, to minimize any variety impact, and milling results indicated no meaningful difference in flour yield or degree of refinement. The major finding of this study indicated a significant increase in the number of specks/image with increasing fusarium damage. As anticipated, each patent flour displayed fewer


specks than its corresponding straight-grade flour noodle, yet the regression lines for the two flour levels were almost parallel against the fusarium levels when read after 2 h. The significant linear relationship between the number of detectable specks and fusarium damage levels in the wheat remained significant (r 2 = 0.61 and 0.63) when the noodles were aged for 24 h. White salted noodles prepared from patent flours did not display any significant relationship between speck numbers and the degree of fusarium damage at 2 h post-production. However, noodles prepared from the straightgrade flour revealed a very significant (r 2 = 0.65, P < 0.025) relationship at this time. Fusarium damage levels were found to influence significantly the number of specks in both patent and straight-grade flour after 24 h (r 2 = 0.61 and 0.63, respectively).

3.3 Frost damage

Mean speck count at 24 h/25cm2

In Canada and most northern US states, a premature frost can have a detrimental impact on the wheat crop. The impact of the frost is usually a much harder kernel than normal, which during the milling process results in a poor mill yield and subsequently poorer flour color. Yellow alkaline and white salted noodles made from increasingly frostdamaged wheat are seen in Figure 21.9. In both cases, as the quality of the material deteriorates, moving from a high quality No.1 grade to a poor Feed grade sample, the number of specks were found to increase. 160 120 80 40 0

1H

1L

2H

3M

3L Feed

1H

1L

2H 2M 2L 3H 3M Increasing frost damage

3L Feed

2M

2L

3H

(a)

Mean speck count/25cm2

160

120

80

40

0 (b)

Figure 21.9 Impact of frost-damaged wheat on the appearance of (a) white salted and (b) yellow alkaline noodle sheets (25 cm2 ) 24 hours after production.

Developments and further applications 539

4 Developments and further applications 4.1 Use of flat-bed scanners Previous investigations of noodle appearance and quality had been confined to the use of a macroscope fitted with an RGB camera. Research by Hatcher et al. (2004) removed this constraint and made the use of image analysis available to most noodle researchers by demonstrating the practicality of employing conventional, inexpensive flat-bed scanners (Figure 21.10). Eight wheat cultivars from Canada Prairie Spring Red (CPSR) and ten from Canada Prairie Spring White (CPSW) were milled to a constant 74 percent extraction rate, and made into both yellow alkaline and white salted noodles. The appearance of these noodles (red and white seed coats) was assessed by two identical scanners which employed different color correction algorithms. Significant differences in the number of specks were observed between the two noodle types in either scanner-based system. The immediate benefit of employing the scanner-based system over that of the macroscope was that a much larger, 5 × 5-cm portion of the noodle could be scanned in one pass, thus improving the overall representation of the samples within the same time period as before. The first system software employed the same technique as used in the macroscope system, whereby its gray level was standardized against the Eastman Kodak No. 3 graylevel patch by dividing the sum of each R, G, and B color plane by three. This mean gray level was used to determine the adjustment value to shift the image to a constant reference gray level. No corrections were made for difference due to the scanner’s color sensor, and all other processing algorithms were not modified. The second system was designed for noodle-color measurement and employed the use of a Kodak Q60R1 Color Input Target (Kodak No. 190 7914, Eastman Kodak,

Figure 21.10 An inexpensive flat-bed scanner linked to a modern laptop computer offers ease of use and mobility to noodle manufacturers and their quality control team.


Rochester, NY) in which histograms in each of the RGB color planes were matched to a reference image of the Q60 chart. The output from the histogram-matching process was applied to the noodle-sample images, and a color measurement module was incorporated. Logarithmic optical density measurements of a calibrated Kodak gray scale demonstrated that while there were slight differences between systems, each displayed a strong linear relationship between log optical density and gray level. The results from both scanner systems were constant with previous findings regarding the number of specks in yellow alkaline noodles. At all combinations of gray and minimum threshold size, they consistently displayed a significantly (P < 0.0001) greater number of specks/image in noodles prepared from the red seed-coated material versus white seed-coat flour. Furthermore, in all combinations the number of specks observed increased significantly (P < 0.05) with aging. While white salted noodles revealed fewer specks than their alkaline counterparts, similar to previous studies, they also displayed an increase with aging and a highly significant (P < 0.0001) influence of seed-coat color. The inexpensive scanner systems also retained the ability to provide very discriminating darkness-distribution profiles, which allowed the differences between red and white seed-coated wheat noodles to be quantified. The color measurement system (No. 2) yielded broader distribution profiles for both red and white wheat noodlespecks compared with system No. 1 due to its correction algorithm, which shifted and redistributed the spectral histograms for each of the RGB image components. This offers the analyst, either scientist or noodle manufacturer, closer color discrimination to that of the consumer. In the comparison of red and white seed-coat flour noodles, maximum discrimination was noted for the blue and green components, with fewer differences being contributed by the red region. The blue component at both 2 and 24 h post-production had the greatest impact on assessing noodle-speck darkness. A further advantage of the newer color-based system was its ability to provide a color measurement of the noodle surface without including the influence of the specks. It thus allowed the impact of the change in the base noodle matrix color to be measured independently – which is not an option in existing colorimeters.

4.2 Impact of the addition of functional food additives Barley, which has a significant history in food use, is now being recognized for its beneficial health properties. It is rich in total dietary fiber, such as b-glucans, arabinoxylans, cellulose, fructans, and arabinoglycans. The b-glucans are very noteworthy, as they have demonstrated beneficial reductions in plasma cholesterol, the glycemic index, and colon cancer (Anderson et al., 1990; Jadhav et al., 1998; Slavin et al., 2000). Image-analysis assessments of the impact of adding barley flours derived from materials with different and diverse amylose content on noodle appearance have been undertaken (Hatcher et al., 2005; Izydorczyk et al., 2005; Lagasse et al., 2006). Hullless barley flour was added to white-coated CPSW patent flour at both 20 and 40 percent levels prior to the production of alkaline noodles. The addition of the barley flour from any of the eight barley cultivars resulted in a significant increase in the number of observed specks, representing a 15- to 20-fold increase over the control CWRS

Developments and further applications 541

patent flour (Hatcher et al., 2005) as early as 1 h after production. With aging (24 h) and concurrent darkening of the background noodle matrix, the noodles prepared using any barley flour continued to display a significantly greater number of specks than the control, although the difference was not as great as immediately after production. It was not surprising to observe that the number of specks detected increased with increasing barley flour addition. The barley fragment specks, consisting primarily of testa, endosperm cell wall, and aleurone layers, as determined by microscopic examination, were found to be significantly larger than those of the control at both 1 and 24 h. White salted noodles were also examined (Lagasse et al., 2006) and, as previously observed, resulted in significant increases in the number of specks detected. Significant differences in the degree of speckiness, however, were observed on the basis of the barley cultivar employed, with the variety CDC Candle being speckier at each time measurement. In contrast, the cultivar SB94792, a zero-amylose waxy barley, consistently showed fewer specks than the remaining six lines. In all cases, the number of specks detected in the salted noodles was significantly lower than in their corresponding alkaline noodles. While at 1 and 2 h post-production the barley-supplemented noodles displayed larger speck size than did the wheat-only control, by 24 h no difference in speck size was observed. The health benefits of barley can be found primarily in their fiber-rich fractions, which are the coarse discarded material derived from the shorts duster at the end of the mill flow. Incorporation of this milling fraction significantly alters the appearance of noodles, and the common consumer evaluation criteria for appearance are overridden by their health-conscious appeal. Incorporation of this fraction into the wheat flour required more water in their production, but excellent noodle texture was achieved. The presence of this health-benefiting fraction in either alkaline or white salted noodles increased both the number of specks and their size, relative to those in the patent wheat control noodles, approximately 150-fold (Izydorczyk et al., 2005). While no differences were observed between the source of barley-fiber rich fractions on either speck numbers or size, the specks detected in salted noodles were five to seven times fewer and significantly smaller than those in their alkaline counterparts.

4.3 Use of artificial neural networks Recent advances have been made in the use of incorporating scanner-based systems to yield the same information as standard colorimeters (Shahin et al., 2006) in their assessment of noodle appearance. An artificial neural network developed for lentils (Shahin and Symons, 2001) and adapted for wheat (Shahin and Symons, 2003) has proven to be very effective in the assessment of oriental noodles. These authors have been able to relate image color histograms derived from inexpensive and independent scanners to the CIE L∗ (brightness), a∗ (red–green) and b∗ (yellow–blue) values of a colorimeter. Previous research by Black and Panozzo (2004) as well as Cho et al. (1995) had demonstrated the benefits of neural networks for converting R, G, and B (red, green, and blue) or H, L, and S (hue, lightness, and saturation) to a device-independent color space. Shahin et al. (2006) were able using alkaline noodle measurements (1648 images) over a 24-h period to establish an r 2 of 0.994 between RGB measurements


and HunterLab colorimeter L∗ readings for a neural network training set. The results remained consistent, r 2 = 0.983, when tested on an independent validation set. The relationship established for noodle redness (a∗ ) was also excellent, r 2 = 0.965, for the training set, but dropped to r 2 = 0.877 on an independent validation set. The assessment of noodle b∗ (yellow–blue) also remained very significant, with r 2 = 0.979 for the training set and 0.878 for the validation set.

5 Conclusions Image analysis of noodles offers manufacturers a variety of benefits, the least of which is to have a physical image of their product for each production run that does not deteriorate over time. On a more quality-conscious basis, image analysis offers the manufacturers the ability to remove subjective visual inspection from the quality-control process and replace it with an objective measure. The technique also offers the manufacturers a number of financial benefits, as it has been demonstrated to detect differences in noodle quality on the basis of wheat variety, wheat class, growing location, degree of flour refinement, and the presence/impact of common grading factors. This allows manufacturers to source and evaluate potential wheat or flours from different sources under their own specific manufacturing process to determine quality effects. The ability to change gray settings, minimum speck threshold size, and to now record L∗ , a∗ , and b∗ noodle-sheet values independently or in combination with specks, allows very precise quality control at a fraction of the cost of common scientific equipment. Large-scale manufacturers in Asia often have more than one production facility either within the country or across countries. This technique allows direct comparison and quality assurance across all facilities, with the results being sent to the central quality center electronically. Throughout Asia manufacturers take a great deal of pride in announcing their ISO certification, normally ISO 9002 quality management. The use of image analysis offers the progressive manufacturers the opportunity to claim ISO 1702-5 accreditation as well. Image analysis of noodles has come full circle. Initial equipment based upon macroscopes and CCD cameras was too expensive for manufacturers to implement. However, with the advent of inexpensive flat-bed scanners and powerful mobile laptop computers, there is no reason for manufacturers not to embrace this technology. Future developments in the image analysis of noodles, however, will require a return to more expensive research and development equipment. Perhaps the most promising aspect lies in the use of multispectral or hyperspectral image analysis. The use of a wavelength band expanded into the infrared region has provided glimpses of further unique abilities available for noodle quality assessment.

References Ambalamaatil S, Lukow OM, Hatcher DW, Dexter JE, Malcolmson LJ, Watts BM (2002) Milling and quality evaluation of Canadian Hard White Spring Wheats. Cereal Food World, 47, 319–328.

References 543

Anderson JW, Deakins DA, Floore TL, Smigh BM, Whitis SE (1990) Dietary fiber and coronary heart disease. Critical Reviews in Food Science and Nutrition, 29, 95–147. Anonymous (1999) Group says ramen demand to double by 2010. Milling and Baking News, 77 (51), 8. Black CK, Panozzo JF (2004) Accurate technique for measuring color values of grain and grain products using a visible NIR instrument. Cereal Chemistry, 81, 469. Chen PMT (1993) Noodle manufacturing technology. In Grains and Oilseed: Handling, Marketing, Processing, Vol. II. Winnipeg: Canadian International Grains Institute, pp. 809–830. Cho MS, Kang BH, Lou MR (1995) Device calibration of a color image scanner digitizing system using neural networks. Proceedings of the IEEE International Conference on Neural Networks, 1, 59. Francis FJ, Clydesdale FM (1975) Food Colorimetry: Theory and Applications. Westport: Avi. Hatcher DW (2001) Asian noodle processing. In Cereals Processing Technology (Owens G. ed.). Cambridge: CRC Press, Woodhead Publishing Ltd, pp. 131–157. Hatcher DW, Kruger JE (1993) Distribution of polyphenol oxidase in flour millstreams of Canadian common wheat classes milled to three extraction rates. Cereal Chemistry, 70, 51–55. Hatcher DW, Kruger JE (1996) Simple phenolic acids in flours prepared from Canadian wheat: relationship to ash content, color and polyphenol oxidase activity. Cereal Chemistry, 74, 337–343. Hatcher DW, Symons SJ, Kruger JE (1999) Measurement of the time-dependent appearance of discolored spots in alkaline noodles by image analysis. Cereal Chemistry, 76 (2), 189–194. Hatcher DW, Symons SJ (2000a) Assessment of oriental noodle appearance as a function of flour refinement and noodle type by image analysis. Cereal Chemistry, 77 (2), 181–186. Hatcher DW, Symons, SJ (2000b) Image analysis of Asian noodle appearance: impact of hexaploid wheat with a red seed coat. Cereal Chemistry, 77 (3), 388–391. Hatcher DW, Symons SJ (2000c) Influence of sprout damage on oriental noodle appearance as assessed by image analysis. Cereal Chemistry, 77 (3), 380–387. Hatcher DW, Anderson MJ, Clear RM, Gaba DG, Dexter, JE (2003) Fusarium head blight: effect on white salted and yellow alkaline noodle properties. Canadian Journal of Plant Science, 83, 11–21. Hatcher DW, Symons SJ, Manivannan U (2004) Developments in the use of image analysis for the assessment of oriental noodle appearance and color. Journal of Food Engineering, 61, 109–117. Hatcher DW, Lagasse S, Dexter JE, Rossnagel B, Izydorczyk M (2005) Quality characteristics of yellow alkaline noodles enriched with hull-less barley flour. Cereal Chemistry, 82 (1), 60–69. Hatcher DW, Lukow OM, Dexter JE (2006) Influence of environment on Canadian Hard White Spring Wheat noodle quality. Cereal Food World, 51, 184–190. Huang S, Morrison WR (1988) Aspects of protein in Chinese and British common (hexaploid) wheats related to quality of white and yellow Chinese noodles. Journal of Cereal Science, 8, 177–187.


Hutchings JB (1994) Food Color and Appearance. Glasgow: Blackie Academic and Professional. Izycorczyk MS, Lagasse SL, Hatcher DW, Dexter JE, Rossnagel, BG (2005) The enrichment of Asian noodles with fiber-rich fractions derived from roller milling of hull-less barley. Journal of Science and Food Agriculture, 85, 2094–2104. Jadhav SJ, Lutz SE, Ghorpade VM, Salunkhe DK (1998) Barley: chemistry and value-added processing. Critical Reviews in Food Science, 38, 123–171. Kruger JE, Hatcher DW, Dexter JE (1995) Influence of sprout damage on oriental noodle quality. In PreHarvest Sprouting in Cereals (Noda K and Mares DJ, eds). Osaka: Center for Academic Societies, pp. 9–18. Lagasse SL, Hatcher DW, Dexter JE, Rossnagel BG, Izydorczyk MS (2006) Quality characteristics of fresh and dried white salted noodles enriched with flour from hull-less barley genotypes of diverse amylose content. Cereal Chemistry, 83 (2), 202–210. Lambrecht HS (1995) Sulfite substitutes for the prevention of enzymatic browning in foods. In Enzymatic Browning and Its Prevention (Lee C and Whitaker JR, eds). Washington, DC: ACS, pp. 313–323. Mayer AM, Harel E (1991) In Food Enzymology (Fox PF, ed.). London: Elsevier Applied Sciences, pp. 373–398. Miskelly DM (1993) Noodles – a new look at an old food. Food Australia, 45, 496–500. Munck L, Feil C, Gibbons GC (1979) Analysis of botanical components in cereals and cereal products: a new way of understanding cereal processing. In Cereals for Food and Beverages (Inglett GE, Muck L, eds). New York: Academic Press, pp. 27–40. Nagao S (1996) Processing technology of noodle products in Japan. In Pasta and Noodle Technology (Kruger E, Matsuo RB, Dick JW, eds). St Paul: American Association of Cereal Chemists, pp. 169–194. Osuga D, van der Schaaf A, Whitaker JR (1994) In Protein Structure–Function Relationships in Foods (Yada RY, Jackman RL, Smith JL, eds). Glasgow: Blackie Academic and Professional. Pierpoint WS (1969). o-Quinones formed in plant extracts: their reactions with amino acids and peptides. Biochemistry Journal, 112, 609–616. Roberts WA (2002) An emerging market. Prepared Foods, 171, 11–14. Shahin MA, Symons SJ (2001) A machine vision system for grading lentils. Canadian Biosystems Engineering, 43, 7. Shahin MA, Symons SJ (2003) Color calibration of scanners for scanner-independent grain grading. Cereal Chemistry, 80, 285. Shahin MA, Hatcher DW, Symons SJ (2006) Use of imaging methods for assessment of Asian noodle color. Cereal Food World, 51, 172–176. Slavin J, Marquart L, Jacobs D (2000) Consumption of whole grain foods and decreased risk of cancer: proposed mechanisms. Cereal Foods World, 45, 54–58. Symons SJ, Dexter JE (1991) Computer analysis of fluorescence for the measurement of flour refinement as determined by flour ash content, flour grade color and tristimulus color measurements. Cereal Chemistry, 68 (5), 454–460. Uhl S (1997) Ethnic side dishes: the main attraction. Food Product Description, 7, 83–84. Zawistowski J, Biliaderis CG, Eskin NAM (1991) In Oxidative Enzymes in Foods (Robinson DSR, Eskin NAM, eds). London: Elsevier Applied Sciences, pp. 217–273.

Quality Evaluation and Control of Potato Chips and French Fries Franco Pedreschi1 , Domingo Mery2 and Thierry Marique3 1 Universidad de Santiago de Chile, Departamento de Ciencia y

Tecnología de Alimentos, Av. Ecuador 3769, Santiago de Chile, Chile 2 Pontificia Universidad Católica de Chile, Departamento de Ciencia de la Computación, Av. Vicuña Mackenna 4860 (143), Santiago de Chile, Chile 3 Centre pour l’Agronomie et l’Agro-Industrie de la Province de Hainaut (CARAH), 7800 Ath, Belgium

1 Introduction Potato chips have been popular salty snacks for 150 years, and retail sales in the US are worth about $6 billion/year, representing 33 percent of the total sales of this market (Garayo and Moreira, 2002; Clark, 2003). In 2001 about 50 percent of the US potato crop was processed to produce 11 300 million kg of processed potatoes, of which 21.6 percent were made into chips. The worldwide trade over recent years indicates that about 7.4 × 107 kg of potato chips were exported, with a value of ∼$165 million annually (Economic Research Service, 2004). Frying in hot oil at temperatures between 160◦ and 180◦ C is characterized by very high drying velocities, which are critical to improve not only the mechanical but also the structural properties of the potato chips (Baumann and Escher, 1995). Potato chips are thin slices whose moisture content decreases from around 80 percent to almost 2 percent when they are fried. However, the drying in oil inevitably leads to a considerable oil uptake of around 35 percent, most of which is located on the surface of the chip (there is almost no penetration during frying) and adheres to the surface at the end of frying. Therefore, a high proportion of oil penetrates into the food microstructure during the post-frying cooling stage (Ufheil and Escher, 1996; Aguilera and Gloria, 2000; Bouchon et al., 2003). In the potato chip industry, the quality of each batch of potato tubers must be tested before processing, and of the various quality criteria the visual aspect, especially the Computer Vision Technology for Food Quality Evaluation ISBN: 978-0-12-373642-0


22

546 Quality Evaluation and Control of Potato Chips and French Fries

color, is of great importance (Marique et al., 2005). Color of potato chips is the first quality parameter evaluated by consumers, and is critical in the acceptance of the product (Pedreschi et al., 2006). Consumers tend to associate color with flavor, safety, storage time, nutrition, and level of satisfaction, due to the fact that color correlates well with physical, chemical, and sensorial evaluations of food quality. The color of potato chips changes during frying, as the components of potatoes are restructured. Consequently, the surface color reflects not only the heterogeneous surface formed as a result of frying but also the non-homogeneous oil distribution. Visual aspects, such as surface color and appearance, can be studied using computer vision techniques in order to determine the potato chip. Acrylamide, which is formed in potatoes during frying and is highly related to the color of potato chips, is suspected to have critical implications for human health, since it has recently been found to be a carcinogen in rats (Mottram and Wedzicha, 2002; Rosen and Hellenäs, 2002; Stadler et al., 2002; Pedreschi et al., 2005). Potato chip color is affected by the Maillard reaction, which depends on the content of reducing sugars, amino acids, or proteins at the surface. It is also affected by the frying temperature and time (Márquez and Añón, 1986). Generally, potato tubers that contain more than 2 percent of reducing sugars are discarded for frying, since they generate too dark a coloration. Research has demonstrated that 2.5–3 mg of reducing sugar per gram of potatoes should be the maximum value accepted for potato chip preparation (Lisinska and Leszczynski, 1989). In European factories, some computer vision systems are used for the on-line evaluation of potato chips, allowing chips to be sorted according to defects such as black spots or blisters (Marique et al., 2005). Some researchers have been also working on a promising device that is able both to classify chips according to color and to predict acrylamide levels using neural networks (Marique et al., 2003, 2005), and some are currently evaluating devices based on this system with the expectation that it will be fully operational very soon. Apart from the neural network, there is another device based on statistical pattern recognition for color classification of potato chips (Marique et al., 2003; Pedreschi et al., 2004). Researchers in this topic are routinely providing classical visual evaluation against a standard chart, and have conducted a good amount of work to testify which criteria (overall appearance, heterogeneity, contrasted extremities, etc.) should be taken into account by the operator to evaluate the surface of potato chips. In this chapter, the application of computer vision to study the quality attributes of potato chips is summarized.

2 Computer vision Computer vision (CV) is a novel technology for acquiring and analyzing an image in order to obtain information or to control processes. Basically, a computer vision system (CVS) consists of a video camera for image acquisition, illuminants with standard settings, and computer software for image analysis (Papadakis et al., 2000; Brosnan and Sun, 2004). Image processing and image analysis are at the core of CV, with numerous

Computer vision 547

B G R

I

Preprocessing Image acquisition

B G R

Red features R

Segmentation

S

color images

camera

chip

gray image

Green features G

Blue features B

L*a*b* features L

Classification

Intensity features I

binary image

Geometric features γ

“class A”

Figure 22.1 Schematic representation of the pattern-recognition process required for automatic classification of potato chips. (Reprinted from Pedreschi et al., 2004©, by courtesy of the Institute of Food Technologists.)

algorithms and methods being capable of objectively measuring and assessing the appearance quality of agricultural products (Sun, 2004). Figure 22.1 shows a schematic representation of a general CV pattern-recognition process required for the automatic classification of potato chips, which involves the following four steps (Castleman, 1996; Mery et al., 2003): image acquisition, image pre-processing and segmentation, feature extraction, and classification.

2.1 Image acquisition A digital image of the potato chip is captured and stored in the computer. When acquiring images, it is important to consider the effect of illumination intensity and the orientation of specimens relative to the illumination source, since the gray level of the pixels is determined not only by the physical features of the surface but also by these two parameters (Peleg, 1993; Chantler, 1995). Typically, a color digital camera provides an image of which each pixel is associated with three digital values as red (R), green (G), and blue (B). Figure 22.2 shows an image-acquisition system implemented by Pedreschi et al. (2004) to measure the different quality attributes in potato chips. This system is composed of: 1. A color digital camera with 4 megapixels of resolution (Power Shot A70, Canon, Tokyo, Japan) 2. Four natural daylight 18-W fluorescent lights (60 cm in length) with a color temperature of 6500 K (Philips, Natural Daylight, 18 W) and a color index (Ra) close to 95 percent for proper illumination 3. A wooden box where the illuminating tubes and the camera are placed; the interior walls of the box are painted black to minimize background light.


Lamps

Camera

Wooden box Sample

Figure 22.2 Image-acquisition system developed to evaluate potato chip quality. (Reprinted from León et al., 2006©, courtesy of Elsevier.)

2.2 Image pre-processing and segmentation The digital images taken must be pre-processed to improve their quality before they are analyzed. Using digital filtering, the noise in the image can be removed and the contrast enhanced. Sometimes in this step the color image is converted to a gray-scale image, called the intensity image (I). The intensity is used to divide the images into disjointed regions with the purpose of separating the region of interest from the background. This segmented image (S) is a binary image, where 0 (black) and 1 (white) indicate background and object, respectively. In our case, such a region of interest corresponds to the area where the potato chip is located for the test. Segmentation is an essential step in computer vision, and the accuracy of this operation is critical in automatic pattern recognition for food image analysis. This is because pattern recognition is based on the data subsequently extracted from the segmentation process (Brosnan and Sun, 2004). Segmentation detects regions of interest inside the image, or structural features of the object, and can be achieved by three different techniques: thresholding, edge-based, and region-based (Sonka et al., 1998; Sun, 2000) segmentation. Mery and Pedreschi (2005) developed a robust algorithm implemented in Matlab 6.1 software (The MathWorks, Inc., Natick, Mass., USA.) to segment potato chips from the background. The segmentation has three steps: 1. Computation of a high-contrast gray-value image from an optimal linear combination of the RGB components 2. Estimation of a global threshold using a statistical approach 3. A morphological operation in order to fill the possible holes presented in the binary image (Figure 22.3).

2.3 Feature extraction Image feature extraction is one of the most active research topics in computer vision (Du and Sun, 2004). Following segmentation, feature extraction is concentrated principally

Computer vision 549

(a)

(b)

(c)

Figure 22.3 Potato images: (a) color image of a potato chip; (b) gray-scale image of (a); (c) segmented image of (a). (Reprinted from Pedreschi et al., 2006©, by courtesy of Elsevier.)

around the measurement of the geometric properties (size and shape) and surface characteristics of regions (color and texture) (Zheng et al., 2006). It is important to know in advance which features provide relevant information for the classification. In order to reduce the computational time required in the pattern-recognition process, it is necessary to select the features that are relevant for the classification. For this reason, feature selection must be performed in a training phase. Features extracted from potato chip images by Pedreschi et al. (2004) are described in Table 22.1, and are grouped into six types: 1. 2. 3. 4. 5. 6.

Geometric (γ) Intensity (gray-scale image) (I) Red component (R) Green component (G) Blue component (B) Mean values of the L*a*b* components (L).

The details regarding how these features are calculated can be found in the references cited in Table 22.1.

2.4 Classification The extracted features of each region are analyzed and assigned to one of the defined classes. A classifier is designed following supervised training. Simple classifiers can be implemented by comparing the measured features with threshold values. However, it is also possible to use more sophisticated classification techniques, such as statistical and geometric analyses, neural networks, and fuzzy logic (Castleman, 1996; Jain et al., 2000; Mery et al., 2003). For example, in statistical pattern recognition, classification is performed using the concept of similarity – i.e. patterns that are similar are assigned to the same class (Jain et al., 2000). In other words, a sample is classified as class “i” if its features are located within the decision boundaries of class “I”. Furthermore, a decision-tree classifier can be implemented to search for the feature that can separate one class from the other classes with the most confidence (Safavian and Landgebe, 1991).


Table 22.1 Extracted features from images of potato. Type

Feature

Description

Reference

γ γ γ γ γ γ γ γ γ γ

(¯i, ¯j) h, w, A, L, R φ1 . . . φ7 |DF 0 |. . .|DF 7 | FM1 . . . FM4 FZ 1 . . . FZ3 (ae , be ) ae /be α (i 0 , j 0 ) Gd

Center of gravity Height, width, area, roundness, and perimeter Hu’s moments Fourier descriptors Flusser and Suk invariant moments Gupta and Srinath invariant moments Major and minor axis of fitted ellipse Ratio of major to minor axis of fitted ellipse Orientation and center of the fitted ellipse Danielsson form factor

Castleman, 1996 Castleman, 1996 Sonka et al., 1998 Zahn and Roskies, 1971 Sonka et al., 1998 Sonka et al., 1998 Fitzgibbon et al., 1999 Fitzgibbon et al., 1999 Fitzgibbon et al., 1999 Danielsson, 1978

I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B I, R, G, B

G C D K1 . . . K3 Kσ K Q Q σQ Q ¯ Q F 1 . . . F15 φ1 . . . φ7 σg2 Tx d

Castleman, 1996 Mery and Filbert, 2002 Mery and Filbert, 2002 Kamm, 1998 Mery and Filbert, 2002 Mery and Filbert, 2002 Mery, 2003 Mery, 2003 Mery, 2003 Mery, 2003 Mery, 2003 Mery, 2003 Sonka et al., 1998 Mery and Filbert 2002 Haralick et al., 1973

I, R, G, B

KL, DFT, DCT

Mean gray value Mean gradient in the boundary Mean second derivative Radiographic contrasts Deviation contrast Contrast based on CLPa at 0◦ and 90◦ Difference between maximum and minimum of BCLP1 ln(Q + 1) Standard deviation of BCLPa Q normalized with average of the extreme of BCLP1 Mean of BCLPa First components of DFT of BCLPa Hu moments with gray value information Local variance Mean (M) and range () of 14 texture featuresb with d = 1, 2, 3, 4, 5 64 first components of the KL, DFT, and DCT transforma

L

L∗ a∗ b∗

Color components of the region

Hunt, 1991; Papadakis et al., 2000

Castleman, 1996

Reprinted from Pedreschi et al., ©2004, by courtesy of the Institute of Food Technologists. γ, geometric features; I, intensity features; R, red component features; G, green component features; B, blue component features, L, L∗ a∗ b∗ features. a CLP: Crossing line profile, gray function value along a line that crosses the region at its center of gravity. The term BCLP refers to the best CLP – in other words, the CLP that represents the best homogeneity at its extremes (Mery, 2003). b The following features are extracted based on a co-occurrence matrix of the whole image of the potato chips: second angular moment, contrast, correlation, sum of squares, inverse difference moment, mean sum, variance of the sum, entropy of the sum, variance of the difference, entropy of the difference, two measures of correlation information, and maximum correlation coefficient, for a distance of d pixels. c The transformation takes a re-sized window of 32 × 32 pixels which includes the middle of the potato chips.

3 Applications 3.1 Sorting of potato chips Recently, the different features of color, size, shape, and texture have been combined for their applications in the food industry, because in this way they increase the performance of the methods proposed. These features can be applied with various kinds of food,

Applications 551

L*: 58.743568 a*: 8.0011638 b*: 26.339806

L*: 38.099905 a*: 13.974827 b*: 19.029009

Figure 22.4 The color of a complete potato chip and a small circular browned region of it in L*a*b* units. (Reprinted from Pedreschi et al., 2006©, courtesy of Elsevier.)

such as in fried potatoes for the detection and segmentation of surface defects, the prediction and characterization of chemical and physical properties, and the evaluation and determination of sensorial characteristics (Pedreschi et al., 2004).

3.1.1 Color The color of potato chips is an extremely important quality attribute and a fundamental criterion for the potato-processing industry, since it is strictly related to consumer perception, the Maillard reaction, and acrylamide formation (Scanlon et al., 1994; Mottram and Wedzicha, 2002; Stadler et al., 2002; Pedreschi et al., 2006). These reasons make it extremely important to have methods for measuring the color of potato chips properly (Pedreschi et al., 2005; León et al., 2006). In image analysis for food products, color is an influential attribute and powerful descriptor that often implies object extraction and identification, and that can be used to quantify the color distribution of non-homogeneous samples (Brosnan and Sun, 2004). The color of fried potatoes is usually measured usually in the unit of L∗ a∗ b∗ , using either a colorimeter or specific image-acquisition and processing systems. Parameter L∗ is the luminance or lightness component, which ranges from 0 to 100, and parameters a∗ (from green to red) and b∗ (from blue to yellow) are the two chromatic components, which range from –120 to 120 (Papadakis et al., 2000). In the L∗ a∗ b∗ space the color perception is uniform, which means that the Euclidean distance between two colors corresponds approximately to the color differences perceived by the human eyes (Hunt, 1991). More recently, potato-chip color has been measured with computer vision techniques (Scanlon et al., 1994; Segnini et al., 1999; Marique et al., 2003; Pedreschi et al., 2004). Computer vision (CV) is used to measure the color of potato chips objectively, as it provides some obvious advantages over a conventional colorimeter – namely, the possibility of simultaneously analyzing the whole surface and the details of the chip, and quantifying characteristics such as brown spots and other appearance defects on the surface (Figure 22.4).


R

G

B

Preprocessing

Segmentation

binary image

Image formation

Fluorescent lamps Camera

Color conversion procedure

L*a*b* values

Figure 22.5 Schematic representation of a computer vision system used to convert color images from RGB to L*a*b* space. (Reprinted from Pedreschi et al., 2006©, by courtesy of Elsevier.)

3.1.1.1 Color models The use of CV for the color assessment of potato chips requires an absolute color calibration technique based on a common interchange format of color data and the knowledge of which features can be best correlated to the product quality. With a digital camera it is possible to register the color of any pixel of the image of the object using three color sensors per pixel, which depend on the color model being used (Forsyth and Ponce, 2003). The most frequently used color model is the RGB model, in which each sensor captures the intensity of the light in the red (R), green (G) and blue (B) spectra respectively. There have been two trends recently in the application of image color for food quality evaluation: one is to carry out a point analysis, encompassing a small group of pixels for the purpose of detecting small characteristics of the object; the other is to carry out a global analysis of the object under the study of the color histogram in order to analyze its homogeneity (Brosnan and Sun, 2004; Du and Sun, 2004). Pedreschi et al. (2006) recently designed and implemented a CV system to measure representatively and precisely the color of highly heterogeneous food materials, such as potato chips, in L∗ a∗ b∗ units from RGB images (Figure 22.5). Since RGB digital cameras obtain information in pixels, León et al. (2006) developed a computational color conversion procedure that allows the obtaining of digital images in L∗ a∗ b∗ color units from the RGB images by testing five models: linear, quadratic, gamma, direct, and neural network. After the evaluation of the performance of the models, the neural network model was found to perform the best, with an error of only 0.96 percent. The network architecture is shown in Figure 22.6. Finally, in order to show the capability of the proposed method, León et al. (2006) compared the color of a potato chip measured by this approach with that obtained by a Hunter Lab colorimeter. The colorimeter measurement was obtained by averaging 12 measurements (at 12 different places on the surface of the chip), whereas the measurement using the digital color image was estimated by averaging all pixels of the surface image. Measurement from the colorimeter was used as the standard measurement.

Applications 553

ˆ* L

R

aˆ * bˆ *

G B

Figure 22.6 Architecture of the neural network used to estimate L∗ a∗ b∗ values from RGB images. (Reprinted from León et al., 2006©, by courtesy of Elsevier.)

Instrument Hunter lab Digital image

(a)

(b)

L* 61.7 66.9

a* 1.7 1.3

b* 26.6 27.3

(c)

Figure 22.7 Estimation of L∗ a∗ b∗ values of a potato chip: (a) RGB image; (b) segmented image by the method proposed by Mery and Pedreschi (2005); (c) color measured in L∗ a∗ b∗ space using a commercial colorimeter and the approach of León et al. (2006). (Reprinted from León et al., 2006©, by courtesy of Elsevier.)

The results are summarized in Figure 22.7, and the error of the CV system was only 1.8 percent. 3.1.1.2 Classification techniques A pattern-recognition approach was used for the classification of potato chips processed under six different conditions, and good classification results were obtained (Pedreschi et al., 2004). Pedreschi et al. (2004) implemented an approach to classifying potato chips using pattern recognition from color images where more than 1500 features were extracted from each of the 60 potato images tested. The feature selection was carried out based on the Sequential Forward Selection (SFS) method (Jain et al., 2000). Finally, 11 features were selected according to their classification attributes. Although samples were highly heterogeneous, classification of the potato chips using a simple classifier and just a few features was able to obtain a very good performance (accuracy ≥ 90 percent) in all cases. These authors showed that pattern-recognition techniques could easily and successfully be applied to classify highly heterogeneous materials such as fried potato chips processed under different conditions, as well as other food products. Marique et al. (2003) used an artificial neuronal network to classify fried potato chips. In this approach, gray-level features of the apex, the center and the base of each potato chip were obtained from a color image in order to determine the quality class


Acrylamide content (µm/kg)

2000

R2 0.9569

1600 1200 800 400 0 10

Acrylamide content 113.77a* 1062.23 5

0 a*

5

10

Figure 22.8 Acrylamide content vs color parameter a∗ of controlled and blanched potato chips (moisture content of ∼1.8%, wet basis) fried at 120, 150, and 180◦ C. (Reprinted from Pedreschi et al., 2005©, by courtesy of Elsevier.)

to which each chip belonged. Using a relatively small number of samples, the authors obtained good agreement with human inspectors, yielding a classification performance of around 90 percent. 3.1.1.3 Color and frying temperature Color has been extensively used for evaluation of the effect of different temperatures on the quality of fried potato chips. The kinetics of color changes in potato slices during frying at four temperatures were investigated using the CV system implemented by Pedreschi et al. (2006). Furthermore, Pedreschi et al. (2005) found a good linear correlation (r 2 = 0.9569) between the acrylamide content of potato chips (moisture content ∼1.8 percent on a wet basis) and their color represented by the redness component a∗ in the range of the temperatures studied. The redness component a∗ is an indicator of nonenzymatic browning; the lower a∗ value, the paler the potato chip (Figure 22.8). As the frying temperature increased from 120◦ to 180◦ C, the resultant chips became redder and darker as a result of non-enzymatic browning reactions that are highly dependent on oil temperature. Blanching reduced the a∗ value of potato chips due to the leaching out of reducing sugars previous to frying, thus inhibiting non-enzymatic browning reactions and leading to lighter and less-red chips. Figure 22.9 shows how the potato chips increased in redness and became darker as the frying temperature increased from 120◦ to 180◦ C. At the same frying temperature, blanching pre-treatment led to paler potato chips after frying. 3.1.2 Texture

Computer analysis of the surface texture of foods is of interest, because it affects the processing of many food products. For instance, there is a known dependence between the oil uptake and the surface properties of fried potatoes (Pedreschi et al., 2000; Bouchon, 2002). Visual textures are generally formed by the interaction of light with a rough surface, such as that of fried potatoes. Scale-sensitive fractal analysis has been applied directly over topographical data sets (heights as a function of position) to

Applications 555

(a)

(b)

(c)

(d)

(e)

(f)

Figure 22.9 Images of potato chips (moisture content of ∼1.8%, wet basis): (a) control fried at 120◦ C; (b) control fried at 150◦ C; (c) control fried at 180◦ C; (d) blanched fried at 120◦ C; (e) blanched fried at 150◦ C; and (f) blanched fried at 180◦ C. Controls are unblanched slices. Blanching treatment was in hot water at 85◦ C for 3.5 min. (Reprinted from Pedreschi et al., 2005©, by courtesy of Elsevier.)

quantify the important changes in the surface texture of potatoes during frying, such as the area-scale fractal complexity (Asfc) and the smooth-rough crossover (SRC). Another way to perform fractal analysis or to quantify the textural properties of a surface is by using the information contained in images (brightness as a function of position), with the advantage that the topography of the sample is not necessarily correlated with the texture of its surface image (Rubnov and Saguy, 1997; Quevedo et al., 2002). Texture has been used in the quality inspection of potato chips. First, textural features are acquired from images taken of the surfaces of a set of potato chip samples by using video cameras. An identical set of samples of potato chips is used to obtain the quality attributes of the samples, using sensory panellists or instruments. Afterwards, learning models (e.g. statistical learning, fuzzy logic, and neural networks) can be set up to correlate the texture features to the potato chip quality. Based on the information obtained from the learning models, the qualities of different categories of potato chips can be predicted by using their texture features from the images (Pedreschi et al., 2004). Different problems involving appearance are associated with frying potato pieces (either slices or strips). One is the presence of defects such as black dots and necrosis. Searching for such defects involves on-line screening and eventual rejection of every defective chip (Marique et al., 2003). Another is the development of dark coloration because of the Maillard reaction between reducing sugars and amino groups (Márquez and Añón, 1986). This must be assessed at the laboratory for every sample in the potato batch intended for processing, because tubers that look perfectly healthy can develop intensive and heterogeneous browning or dark tips, which lead to consumer rejection. Defective batches are refused, to the disadvantage of the producer.


3.2 Browning evaluation Evaluation of browning of samples taken from the frying lines must also be performed frequently at the laboratory, to police the process. The synthesis of acrylamide during frying increases brownness, due to the reaction of asparagine and reducing sugars (Pedreschi et al., 2005). As this reaction pathway is clearly correlated with the Maillard reaction (Mottram and Wedzicha, 2002; Stadler et al., 2002; Pedreschi et al., 2005), it has been proposed by several authors that quick and easy measurement of browning can be performed using image analysis rather than painstaking chromatographic methods (Pedreschi et al., 2005). Acrylamide is suspected to be a molecule with significant toxicological effects – carcinogenic, neurotoxic, and mutagenic (Rosen and Hellenäs, 2002). 3.2.1 Using artificial neural networks (ANN) by CARAH (Centre pour l’Agronomie et l’Agro-industrie de la Province de Hainaut, Belgium)

To estimate the darkening of French fries during frying, a simple frying assay is performed for 3 minutes at 180◦ C on 20 French fries obtained from the central part of 20 different potatoes. Each of the French fries is then assigned a category by visual examination under standard white light (Marique et al., 2003). The assessors build their evaluation with the help of a standard reference card (Figure 22.10), determined from both the overall darkening of individual French fries, and the contrast between the extremities (apex and base) and the center of the fries. Heterogeneous dark coloration is also penalized. There are, of course, problems associated with this subjective procedure. In particular, estimations may vary with the assessor. Even for a given assessor, sample variability can influence results, since narrow distributions tend to be spread over the scale. It is thus of great interest to develop a model that allows reproducible estimation of the color category of French fries (Marique et al., 2003). Artificial neural networks (ANNs) can attain very good performance when used to predict values for complex non-linear systems (Mittal and Zhang, 2000; Wilkinson and

PF 00 Figure 22.10

PF 0

PF 1

PF 2

PF 3

A reading card of French fries for browning categories.

PF 4

PF 5

Applications 557

Yuksel, 1997). Moreover, they are endowed with a broad capacity for generalization, so that they can give useful information for cases that are not part of their training set (Schalkoff, 1997; Wilkinson and Yuksel, 1997; Marique and Wérenne, 2001; Yang et al., 2002). They appear to be the logical choice for achieving successful prediction of the darkening index for fried potatoes. Marique and colleagues (2003) used image analysis to extract gray-level intensities from an image data bank gathered from the routine frying assays of 12 different mealy potato cultivars (Annabelle, Bintje, Cantate, Charmante, Cyclone, Daisy, Farmer, Innovator, Lady Olympia, Liseta, Markies, Victoria). Three values were computed for individual French fries, corresponding to the mean gray values at the apex, center, and base of the specimen, respectively. The ANN is a feed-forward network consisting of three inputs, a hidden layer of four neurons with sigmoid transfer functions and bias (Figure 22.11), and an output layer presenting a single linear neuron with bias that is issued to the estimated value of the color category (from 0 to 4). The ANN was trained with a Lenvenberg–Marquardt algorithm (Schalkoff, 1997) and the output values were compared to the corresponding color categories estimated by human operators, who assigned each of the French fries to a color category ranging from 0 (very pale) to 4 (very dark). The Lenvenberg–Marquardt algorithm gave fast convergence, as is usually the case for small networks. Figure 22.12 shows how the assessor assigns a particular color value to individual French fries. The different color categories are distributed throughout the gray-value scale, and there is partial overlapping. This comes from the fact that when a particular specimen stands exactly

Hidden 1 Apex

Hidden 2 Color class

Center Output Hidden 3 Base

Hidden 4

Figure 22.11 Structure of a two-layer feed-forward artificial neural network with three inputs, four hidden neurons with bias, and one output neuron with bias. (Reprinted from Marique et al., 2003©, courtesy of Institute of Food Technologists.)


300

300 250

250 200

200 150

150 100

100 50

50

Center

Apex 0

0

Figure 22.12 Mean gray values of the center and apex of French fries. The color code indicates the class of color: very pale gray, 0; pale gray, 1; mid-gray, 2; dark gray, 3; black, 4. The ellipsoids contain the two sub-populations of class 1: dark gray, globally darker fries; pale gray, paler fries with contrasted dark ends. (Reprinted from Marique et al., 2003©, courtesy of Institute of Food Technologists.)

between two categories, the assessor will select one at random and either undervalue or overvalue it, which hence leads to the overlapping. French fries are assigned to category 0 if they appear both very pale (gray levels over 150) or rather paler at the extremities than in the center. A specimen will be assigned to category 1 for one of the following two reasons: it appears paler in the center but has contrasting dark ends (global appearance), or it appears dark in the center with paler ends. Category 1 is thus clearly split in two subpopulations (see ellipsoids in Figure 22.12) flanking both sides of category 0. Categories 2, 3, and 4 then progressively regroup darker French fries, which are generally pale in the center with more or less contrasted dark ends. Thus, it is only for the two lower color categories that the assessor will overvalue a specimen possessing dark contrasted extremities. For higher color categories, estimations are based mostly on the global (center) appearance of French fries, where dark contrasted ends are considered to be “normal” (Marique et al., 2003). The trained ANN was used to generate a complete set of predictions for the different possible combinations of gray levels of the center and the apex of the French fries. This is illustrated in Figure 22.13, where computation was performed using equal gray values for both the base and the apex of the fries. The ANN showed a good performance, with correlation coefficients of 0.972 for the training data and 0.899 for the validation data. The network displays complex and continuous behavior for the low color categories 0, 1, and 2, but operates a discrete classification between categories 3 and 4. This could be a

Applications 559

4 3.5 3 2.5 2

6

1.5 Color class

4 0

2

1 0.5

0

100

2 0

0 0.5

200 100

1

200 300

300

Center

Apex Figure 22.13 Response of the artificial neural network: color class categories are described as a function of all possible combinations of gray levels of the center and the apex of French fries. (Reprinted from Marique et al., 2003©, by courtesy of Institute of Food Technologists.)

consequence of both the greatest number of data points for high color categories, and the more complex behavior of the assessor for low color categories (Marique et al., 2003). A more complete simulation is shown in Figure 22.14, illustrating the discrete color categories (the values predicted from the ANN are approximated by the nearest integer) obtained for all the possible combinations of gray levels of the three regions of the fries. Again, more complex behavior is observed for the lower color categories. The intermediate categories 2, 3, and 4 also extend between two “wings,” being either globally paler with dark ends or globally darker with pale ends. Color classification varies most with the apex and center gray values (Marique et al., 2003). 3.2.2 Using other methods (Walloon Agricultural Research Center, Belgium)

This research center, affiliated to the GemblouxAgricultural University in Belgium, has developed a home-made system for the quality evaluation of French fries (Figure 22.15). In the system, 20 French fries are cooked and arranged on a tray with a reference tongue (Figure 22.16). The reference tongue is directly extracted from the USDA reference card in order to represent the seven reference colors on each sample. For each image, a tray contains 20 French fries and a reference tongue. The tray is then placed in the box and an image is taken. Image analysis is performed using Image Pro Plus software 6 (Media Cybernetics, USA). The program checks the number of references, which are represented by the seven colored squares arranged on each tray, and offers a manual correction in cases

300

300

250

250

200

200 Apex

Apex


150

100

50

50

300

250

200

(a)

150 100 Center

50

200 400 Base

0 300

250

200

200

100

50

50

250

200

(c)

150 100 Center

50

300

200 300 0 100 Base

300

200 Apex

200

100

50

50

200

150 100 Center

50

200 300 0 100 Base

200

150 100 Center

50

4

0 300

(f)

200 300 0 100 Base

150

100

250

250

300 250

300

200 300 0 100 Base

2

(d)

250

0

50

0

3

150

150 100 Center

150

100

300

200

300

250

150

250

(b)

1

0

Apex

0

Apex

Apex

300

(e)

150

100

0

0

250

200

150 100 Center

50

200 300 0 100 Base

Figure 22.14 Discrete color categories obtained from all the possible combinations of gray levels for the three regions of the French fries: (a) general presentation; (b)–(f) partial views of each color category, from 0 (b) to 4 (f). (Reprinted from Marique et al., 2003©, by courtesy of Institute of Food Technologists.)

where this number is different from seven. The references are then scanned. The French fries are also scanned and counted. Once this has been done, macro sequences successively comply in a few seconds (via sending data to Excel) to compare the colors of the French fries with the reference tongue in order to determine the color parameters. For this comparison, RGB images are first converted into gray-scale and their luminous intensity is then estimated. These stages are repeated until there is automatic generation of a detailed written (containing photographs of the sample, the date, color parameters etc.) and archived report.

Applications 561

Figure 22.15 Device for quality evaluation of French fries (Gembloux Agricultural University in Belgium).

Figure 22.16 Tray ready for image analysis with 20 French fries and 7 colored squares.

In order to compare the image analysis method with sensory analysis (visual comparison), a series of 100 samples is first analyzed using both methods. The correlation of the results between the two methods is 0.951, which is a very good result compared with the weak repeatability of the reference method (Figure 22.17). 3.2.3 Browning-sorting and acrylamide estimation using ANN by CARAH

Determination of the acrylamide concentration nowadays appears to be necessary since very high concentrations of this potentially toxic molecule are detected in amylaceous


6.00 Judge index 0.7491(Image analysis index) 0.6221 R 0.9510 5.00

Judge index

4.00

3.00

2.00

1.00

0.00 0.00

Figure 22.17

1.00

2.00 3.00 4.00 Image analysis index

5.00

6.00

Correlation between the visual measurements and those by image analysis.

Figure 22.18 Artificial neural network imaging system for the determination of color class and acrylamide concentration in French fries.

fried foodstuffs (Rosen and Hellenäs, 2002). However, standard procedures for acrylamide determination involve slow and expensive methods of chromatography and mass spectroscopy, and thus cannot be used for routine analysis. It was therefore a logical step to develop alternative techniques based on image analysis of the browning of French fries to measure the acrylamide concentration. It was known that there would be good correlation between non-enzymatic browning development and acrylamide formation, since several studies had reported a strong linear relationship between browning and

References 563

acrylamide accumulation in fries (Mottram and Wedzicha, 2002; Stadler et al., 2002; Pedreschi et al., 2005). CARAH and a Belgian industrial partner, Rovi-Tech s.a. (Presles, Belgium), have developed a high-speed imaging system incorporated with ANN. Snapshots of every one of the French fries tested are taken and then results for both color category and acrylamide concentration are obtained (Figure 22.18). The system is intended to analyze incoming potato batches of pre-fried French fries for quality control in food distribution. The heart of the system is Rovi-Tech’s ILB-25 (Image Learning Box), a very efficient ANN that allows easy and powerful correlations of complex visual data analysis.

4 Conclusions Both gray-scale and color images of potato chips are useful for extracting image features for an appropriate classification. The most relevant features for potato-chip classification are either texture or color from L∗ a∗ b∗ space. Potato chips can be properly classified to obtain very good values, despite their high heterogeneity. The automatic classification methodology proposed for potato chips has a wide range of potential uses. The computer vision system described in this chapter allows determination of the color of potato slices in L∗ a∗ b∗ color space that is transformed from the RGB space in an easy, precise, and objective way. To achieve this, image pre-processing and segmentation is performed. The computer vision system allows easy measurement of color not only over the entire potato chip surface, but also at small, specific regions of interest. Five models that can measure the color in potato chips using the color of each pixel on the target surface, which cannot be accessed with conventional colorimeters, have been built. The best results were achieved by the quadratic and neural network models, both with small errors of close to 1 percent. For both control and blanched potato chips, acrylamide formation decreases dramatically as the frying temperature decreases from 180◦ to 120◦ C. There is a linear correlation between non-enzymatic potato-chip browning quantified by computer vision and the corresponding acrylamide values.

Acknowledgments The authors acknowledge financial support from the FONDECYT Project N◦ 1030411.

References Aguilera JM, Gloria H (2000) Oil absorption during frying of frozen par-fried potatoes. Journal of Food Science, 65, 476–479.


Baumann B, Escher E (1995) Mass and heat transfer during deep fat frying of potato slices. I. Rate of drying and oil uptake. Lebensmittel Wissenschaft und Technologie, 28, 395–403. Bouchon P (2002) Modeling oil uptake during frying. PhD Thesis, Reading University, England. Bouchon P, Aguilera JM, Pyle DL (2003) Structure oil-absorption relationships during deep-fat frying. Journal of Food Science, 68, 2711–2716. Brosnan T, Sun D-W (2004) Improving quality inspection of food products by computer vision – a review. Journal of Food Engineering, 61, 3–16. Castleman K (1996) Digital Image Processing. Englewood Cliffs: Prentice Hall, p. 667. Chantler M (1995) Why illuminant direction is fundamental to texture analysis. IEE Procedures Visual Image Signal Process, 142, 199–206. Clark JP (2003) Happy birthday, potato chip! And other snack developments. Food Technology, 57(5), 89–92. Danielsson PY (1978) A new shape factor. Computing Graph Image Processing, 7, 292–299. Du C-J, Sun D-W (2004) Recent developments in the applications of image processing techniques for food quality evaluation. Trends in Food Science & Technology, 15, 230–249. Economic Research Service (ERS) of the US Department of Agriculture (2004) Briefing room potatoes. Available at http://www.ers.usda.gov./briefing/potatoes/. Fitzgibbon A, Pilu M, Fisher R (1999) Direct least square fitting ellipses. IEEE Transactions on Pattern Analysis, 21(5), 476–480. Forsyth D, Ponce J (2003) Computer Vision: A Modern Approach. Englewood Cliffs: Prentice Hall. Garayo J, Moreira R (2002) Vacuum frying of potato chips. Journal of Food Engineering, 55, 181–191. Haralick RM, Shanmugan K, Dinstein L (1973) Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, 3, 610–621. Hunt RWG (1991) Measuring Color, 2nd edn. Chichester: Ellis Horwood, Ltd, p. 313. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Transactions Pattern Analysis Machines Intelligence, 22, 4–37. Kamm KF (1998) Grundlagen der Röntgenabbildung. In: Moderne Bildgebung: Physik, Gerätetechnik, Bildbearbeitung und -kommunikation, Strahlenschutz, Qualitätskontrolle (Ewwen K ed.). Stuttgart: Georg Thieme Verlag, pp. 45–61. León K, Mery D, Pedreschi F, León J (2006) Color measurements in L∗ a∗ b∗ units from RGB digital images. Food Research International, 39(10), 1084–1091. Lisinska G, Leszczynski W (1989) Potato Science and Technology. London: Elsevier Applied Science. Marique T, Wérenne J (2001) A general artificial neural network for the modeling of culture kinetics of different CHO strains. Cytotechnology, 36, 55–60. Marique T, Kharoubi P, Bauffe P, Ducatillon C (2003) Modelling of fried potato chips color classification using image analysis and artificial neural network. Journal of Food Science, 68, 2263–2266.

References 565

Marique T, Pennincx S, Kharoubi A (2005) Image segmentation and bruise identification on potatoes using a Kohonen’s self-organizing map. Journal of Food Science, 70, 415–417. Márquez G, Añón MC (1986) Influence of reducing sugars and amino acids in the color development of fried potatoes. Journal of Food Science, 51, 157–160. Mery D (2003) Crossing line profile: a new approach to detecting defects in aluminium castings. Proceedings of the Scandinavian Conference on Image Analysis (SCIA 2003) (Bigun J, Gustavsson T, eds). Lecture Notes in Computer Science, 2749, 725–732. Mery D, Filbert D (2002) Classification of potential defects in automated inspection of aluminium castings using statistical pattern tecognition. In Proceedings of the 8th European Conference on Non-Destructive Testing (ECNDT 2002), June 17–21, Barcelona, p. 14. Mery D, Pedreschi, F (2005) Segmentation of color food images using a robust algorithm. Journal of Food Engineering, 66, 353–360. Mery D, da Silva R, Caloba LP, Rebolledo JMA (2003) Pattern recognition in the automatic inspection of aluminium castings. Insight, 45, 475–483. Mittal GS, Zang J (2000) Use of artificial neural network to predict temperature, moisture, and fat in slab-shaped foods with edible coatings during deep-fat frying. Journal of Food Science, 65, 978–983. Mottram DS, Wedzicha BL (2002) Acrylamide is formed in the Maillard reaction. Nature, 419, 448–449. Papadakis SE, Abdul-Malek S, Kamdem RE, Yam KL (2000) A versatile and inexpensive technique for measuring color of foods. Food Technology, 54(12), 48–51. Pedreschi F, Aguilera JM, Brown CA (2000) Characterization of food surfaces using scale-sensitive fractal analysis. Journal of Food Process Engineering, 23, 127–143. Pedreschi F, Mery D, Mendoza F, Aguilera JM (2004) Classification of potato chips using pattern recognition. Journal of Food Science, 69, 264–270. Pedreschi F, Moyano PC, Kaack K, Granby K (2005) Color changes and acrylamide formation in fried potato slices. Food Research International, 38, 1–9. Pedreschi F, León J, Mery D, Moyano P (2006) Implementation of a computer vision system to measure the color of potato chips. Food Research International, in press. Peleg M (1993) Fractals and foods. Critical Review Food Science & Nutrition, 33, 149–165. Quevedo R, López C, Aguilera JM, Cadoche L (2002) Description of food surfaces and microstructural changes using fractal image textura analysis. Journal of Food Engineering, 53, 361–371. Rosen J, Hellenäs KE (2002) Analysis of acrylamide in cooked foods by liquid chromatography tandem mass spectrometry. Analyst, 127, 880–882. Rubnov M, Saguy IS (1997) Fractal analysis and crust water diffusivity of a restructured potato product during deep-fat frying. Journal of Food Science, 62, 135–137, 154. Safavian S, Landgrebe D (1991) A survey of decision tree classifier methodology. IEEE Transactions on Systems, Man, and Cybernetics, 21(3), 660–674. Scanlon MG, Roller R, Mazza G, Pritchard MK (1994) Computerized video image analysis to quantify color of potato chips. American Potato Journal, 71, 717–733.


Segnini S, Dejmek P, Öste R (1999) A low cost video technique for color measurement of potato chips. Lebensmittel Wissenschaft und Technologie, 32, 216–222. Schalkoff RJ (1997) Artificial Neural Networks. New York: McGraw-Hill, p. 448. Sonka M, Hlavac V, Boyle R (1998) Image Processing, Analysis and Machine Vision, 2nd edn. Pacific Grove: PWS Publishing. Stadler RH, Blank I, Varga N, Robert F, Hau J, Guy A, Robert P, Riediker MC (2002) Acrylamide from Maillard reaction products. Nature, 419, 449–450. Sun D-W (2000) Inspecting pizza topping percentage and distribution by a computer vision method. Journal of Food Engineering, 44, 245–249. Sun D-W (2004) Computer vision – an objective, rapid and non-contact quality evaluation tool for the industry. Journal of Food Engineering, 61, 1–2. Ufheil G, Escher F (1996) Dynamics of oil uptake during deep-fat frying of potato slices. Lebensmittel Wissenschaft und Technologie, 29, 640–644. Wilkinson C,Yuksel D (1997) Using artificial neural networks to develop prediction models for sensory-instrumental relationships: an overview. Quality Preferences, 8, 439–445. Yang CC, Prasher SO, Landry J, Ramaswamy HS (2002) Development of neural networks for weed recognition in corn fields. Transactions of the ASAE, 45, 859–864. Zahn C, Roskies R (1971) Fourier descriptors for plane closed curves. IEEE Transactions on Computing, 21, 269–281. Zheng C, Sun D-W, Zheng L (2006) Recent developments and applications of image features for food quality evaluation and inspection – a review. Trends in Food Science and Technology, 17(12), 642–655.

Index

A Acousto-optics tunable filter (AOTF), 168 Acremonium zeae, 416 Acrylamide, 556 Active contour model (ACM), 48 drawbacks, 49 Active mode of sensors, 11 Adhesive-tape method, for preparation of specimens, 384 Air-blast cooling process, 148 Air-bridge isolation integrated circuits, 23 Airsacculitis, 157, 160 lesions in chicken carcasses, 171–172 Algorithm(s) back-propagation (BP) learning, 497 decision-tree, 170 flooding, 225 graphic, 52 separation, 358–359 skeletonization, 457–458 snake, 229–230 watershed, 146 Alternaria citri, 246 ANN. See Artificial neural network Aphelion, 42 Apples categories, 213 production, 213 quality evaluation of, using computer vision. See Apples, quality inspection sorting, using statistical methods, 67–68 Apples, quality inspection, 213–215 backpropagated neural network method, 228 Bayes’ theorem, 228 boundary analysis, 214 calyxes and stalk-ends defect recognition, 230–231 color grading, 222–224

defect recognition and fruit classification, 231–232 color parameter, 232 hierarchical grading based on object clustering and fruit classification, 234–237 hierarchical grading based on object supervised classification, 233–234 texture parameter, 232–233 defect segmentation method, 228–229 defects in, 221 evaluation methods, 224–230 devices to acquisition of images, 216–217 flooding algorithm, 225 image acquisition, 214 devices, 216–217, 220 image database, 220–222 Jonagold apples, defect segmentation, 227–228 lighting system geometry (or repartition), 218–220 spectral composition, 220 manipulation and presentation of images, 215–218 multilayer back-propagated neural network method, 236–237 San Fuji apples, defect segmentation, 228 shape grading, 222 snake algorithm, 229–230 Application programming interface (API), for camera programming, 355 Area ratio, of pizza, 430 Area-scale fractal complexity (Asfc), 555 Area-scan imaging system, 354–355 Arnott test, of meltability, 448 Artificial neural networks (ANNs) for classification of olives, 298–299 of French fries, 556–559 learning process, 84–86 structure, 82–84 Ascites, 157

568 Index

Aspergillus flavus, 416 Aspergillus niger, 416 Aspergillus spp., 246 Atomic force microscopy (AFM), 389 Automated defect detection, of potatoes bruise and green-spot detection, 311–313 on-line sorting, 310–311 Automatic inspection systems, 259 Autoregressive models, 61, 62–63 B Backlighting, citrus fruits inspection, 259 Back-propagation (BP) learning algorithm, 497 Back-propagation network (BPN), 361 Back-propagation neural network (BPNN), 364 Bakery products characteristics of color, 485–492 rheological properties and textural properties, 492–495 computer vision inspection of color, 495–503 shape and size inspection, 503–512 texture inspection, 512–517 EuroAsian market for, 481 presence of dairy products, 486 Ballistic photons, 321 Bayesian classification, 87–88 Bayesian classificator, 298 Bayesian classifier structure, 88 Bayesian decision rule, 87 Bayesian discriminate analysis for olive-grading process, 298 Bayesian probability, 51 Bayesian theory, 51 Bayes’ theorem, for defect detection in apples, 228 BCS. See Beef Color Standard Beef carcasses composition, regression analysis, 120 skeletal maturity, 116–117 yield determination, 118 yield grading, 118–119 images, 114 segmentation of, 52 joints, shrinkage of cooked, 62 marbling scores, 113 quality evaluation carcass composition, prediction, 120–121 quality and yield grades, prediction, 117–119 quality attributes, characterization, 112–117 tenderness, prediction, 121–125 BeefCam, 119–120 Beef Color Standard (BCS), 112 Beef Marbling Standard (BMS), 113

Beef tenderness, 72 measurement methods sensory evaluations, 124–125 Warner-Bratzler shear forces, 122–124 statistical methods for measuring, 67–68 Binary classification, 97–99 Binary scales of images, 52 BMS. See Beef Marbling Standard Bolometer detector, 23 Boltzmann equation, 324 Borland C++ programming language, 503 Boundary analysis, for quality inspection of apples, 214 Boundary-based method boundary analysis and classification, 62–63 autoregressive models, 62–63 Fourier transform, 62 boundary representation, 61 Box-whisker diagrams for olive-sorting process, 297 Bread-crumb texture analysis, 513 Breakage of corn, 413 Bright greenish-yellow (BGY) test, 416 Brine, 290, 295 olives, 291 Bruise and green-spot detection, 311–313 Bruised potatoes, 306 Bruises, defined, 306 Bruising, 157 C C. ferrugineus, 364 Cadaver, 157 Calcified objects, 25 Camera CCD, 11 color, 301 complementary metal oxide silicon (CMOS), 11 indium gallium arsenide (InGaAs)-based near-infrared (NIR), 5 in industrial system for olive sorting, 294 Canada GrainAct (1975), 352 Canadian durum wheat (CWAD), 404 Canny edge detector, 48 Cartilage and bone, image segmentation of, 52 Cartilage ossification, 116 Catfish (Ictalurus punctatus), 206 Cattle muscles, marbling degree, 113 CCD architectures, 12 area arrays, 12 camera, 12–13, 163 high-resolution, 177 imagers, 12 scanner, 12

Index 569

sensor, 11 shift register, 12 C4.5 classifier, 432 Cell-scale micro-structure, of cooked grain, 393–397 Centroid-contour distance method, 505 Ceratitis capitataWied. (Medfly), 246 Chain code, 61. See also Boundary-based method Charge-coupled device (CCD) camera, 324, 326–327 Cheddar cheeses, 450 Cheese browning factor (BF), 489 defects calcium lactate crystals, 462–463 mechanical openings, 463–465 end-use qualities browning and blister formation, 451–453 meltability, 448–451 oiling-off, 453–454 microstructure evaluation analysis of SEM micrographs, 465–466 dynamic 4D microstructure evaluation, 470–474 using confocal laser scanning microscopy (CLSM), 466–470 quality characteristics, 447–448 shred morphology and integrity, 454–462 Chicken carcasses, 171–172 inflammatory processes in, 160 skin tumors, 165–166 systemic problems in, 165 Chirostoma estor estor, 190 CIE. See Commission International de L’Éclairage CIE La*b*, 66 CIELAB system, 166, 486 CIE Lu*b*, 66 CIELUV color, 162 Circularity index, of pizza, 431 Citrus fruits characteristics, physiological and physicochemical, 243–244 defects causing appearance-related problems, 245 Ceratitis capitataWied. (Medfly), 246 fungal infestations, 245–246 production, 243 quality evaluation. See Citrus fruits, quality inspection parameters for inspection, 244–245 Citrus fruits, quality inspection backlighting, 259

clementine and satsuma segments inspection, 258–259 computerized axial tomography (CAT) scanning, 257–258 internal quality inspection, 256–258 by machine vision, 246–247 magnetic resonance imaging (MRI), 256–257 neural networks, 251, 254 non-visible spectrum, 255 hyperspectral vision, 255–256 pixel-oriented techniques, 251, 252 radiowaves (RF) pulse, 257 region-oriented segmentation techniques, 252–253 visible spectrum, 247–248 image acquisition of peel, 249–250 image analysis, 250–254 scene lighting, 248–249 Classification-based segmentation, 49–51 Clostridium tyrobutyricum, 464 CMYK (cyan, magenta, yellow, black) color space, 64 Colletotrichum gloeosporioides, 246 Collimator, 25 Color camera, 301 of cooked meat correlation with water content, 150 measurement, 149 defects in olives, 292 of fried potatoes, 551 grading in apples, 222–224. See also Hierarchical fruit grading method, for apples imaging for bruises detection in strawberries, 274–275 imaging for quality inspection characterizing wholesomeness, 161–162 splenomegaly detection, 159–160 viscera inspection, 160–161 spaces, types of, 63–67 Color-based sensing technique, 162 Colorimetric analysis, 293 Color index (CI), 244 Combination of wavelet features and Gabor features (CWG), for determining image texture, 152–153 Commission International de L’Éclairage (CIE), 65, 268 Compressed cooked rice grain, 395–396 Computer-aided machine vision systems, 379 Computerized axial tomography (CAT) scanning, 257–258

570 Index

Computer vision, 19, 23 applications in food industry, object measurement. See Object measurement methods method for pore characterization, 145–147 system, dual-component, 119 techniques, 142 Computer vision-based studies, for quality inspection of apples, 213–215 backpropagated neural network method, 228 Bayes’ theorem, 228 boundary analysis, 214 calyxes and stalk-ends defect recognition, 230–231 color grading, 222–224 defect recognition and fruit classification, 231–237 defect segmentation method, 228–229 defects evaluation methods, 224–230 flooding algorithm, 225 image acquisition devices, 216–217, 220 lighting system, 218–220 manipulation and presentation of images, 215–218 multilayer back-propagated neural network method, 236–237 shape grading, 222 snake algorithm, 229–230 of citrus fruits clementine and satsuma segments inspection, 258–259 computerized axial tomography (CAT) scanning, 257–258 internal quality inspection, 256–258 magnetic resonance imaging (MRI), 256–257 neural networks, 251, 254 non-visible spectrum, 255–256 pixel-oriented techniques, 251, 252 radiowaves (RF) pulse, 257 region-oriented segmentation techniques, 252–253 visible spectrum, 247–254 of strawberries anthocyanin distribution estimation, 283–284 bruises detection, 273–279 challenges, 284–285 fecal contamination, detection of, 279 firmness and soluble-solids content, estimation of, 279–283 soluble-solids content measurement, 283 Computer vision-based textural analysis, of bakery products, 512–517 Conduction-heating test, for cheese, 450

Confocal laser scanning microscopy (CLSM), 467–470 Consumer preferences, 244–245 Contrast enhancing, 39 histogram equalization, 40 histogram scaling, 40 purpose of, 39 Contrast-limited adaptive histogram equalization (CLAHE), 41, 146 Convolution mask, 69 Co-occurrence matrix, 67–68 Cooked meats color of correlation with water content, 150 measurement, 149 pores and porosity of, 145 correlation with water content, processing time, and texture, 147–149 measurement of, 146–147 shrinkage of. See Shrinkage of cooked meats Cooked pork ham, 149 Cooking efficiency, 148 Cooking time and browning property of food material, 489 Coomassie brilliant blue (CBB), 385 Corn grading, 408–409 quality assessment using machine vision systems breakage, 413 color, 410–411 hardness or vitreousness, 416–417 heat damage, 415 hyperspectral imaging, 418 mold and fungal contamination, 415–416 near-infrared reflectance (NIR) spectroscopy, 418 real-time husk deduction measurements, 418 seed viability, 417–418 separation of shelled corn from residues, 418 size and shape, 412–413 stress cracks, 414 synchrotron Fourier transform infrared (FTIR) microspectroscopy, 418 use of, 408 Couplants, 14 CT, 25 imaging, 26 scanner, 24 Curve-fitting procedure, 331–332 D DDAG. See Decision directed acyclic graph Decision directed acyclic graph (DDAG), 101–102

Index 571

Decision-tree algorithm, 170 Decision tree structure, 94–95 Deformed shape, of food dough, 505 Defuzzification, 94 Delta weight equation, 176 Denaturation of protein, 146 Dewar, 20 Diffuse illuminator, 10 Diffuse photons, 321–322 Diffuse reflectance, 365–366 Diffuse transmittance, 366 Digital cameras, 13 Digital signal processors (DSP), 314 Dimension reduction, 50–51 Diplodia maydis, 416 Disconnect algorithm, 358 Dispersive systems, 366–367 Dual-component computer vision system, 119 Durum wheat, 404, 406 E Edge detection. See Gradient-based segmentation Electrical-resistant tomography (ERT) system, 29–31 Electromagnetic spectrum, 4–6 Electromechanical filter wheels, 168 Electron beam, 22 Ellipse-fitting algorithm, 359 Euler–Cauchy method, 487 European CCIR, 13 F Farrell’s model, of spatial diffuse reflectance, 325, 331–332 Fast Fourier transform (FFT) algorithms, for poultry image classification, 172–173 Fat-ring test protocol, 454 Federal Meat Inspection Act and the Poultry Products Inspection Act, 158–159 Feed forward neural network (FFNN), 497 Feret’s diameter, 58, 429 Fermentation, 294 of olives, 290–291 Fiber-optic spectroscopy, for disease classification in poultry, 171 Field programmable gate array (FPGA) processor, 511–512 First-order geometric moments, of objects, 508 First-order gray-level statistics (FGLS), for determining image texture, 150–152 Fisher discriminant analysis, 88–89 Flesh damage in olives, 292 Flooding algorithm, for defect detection in apples, 225

Flowchart of olive-sorting process, 295 of table olives production process, 290 Food products classification techniques artificial neural network (ANN), 82–86 decision tree, 94–96 fuzzy logic, 91–94 statistical classification (SC), 86–91 support vector machine (SVM), 96–102 surface measurements, types of color, 63–67 textures, 67–71 training set, 95–96 types of geometric measurements shape, 59–63 size, 58–59 Food Safety Inspection Service (FSIS), 157, 165, 169 on fecal contamination, 177 on poultry products, 158–159 Fourier-based shape classification technique, 310 Fourier transforms (FT), 61, 62, 69, 508–509 Fragaria x ananassa Duch. See Strawberries Frame grabber digital, 13 Matrox Meteor RGB, 295 Frame-grabber card, 13 Free-oil formation, in cheese, 453–454 Frequency-domain techniques, 322–323 Fruit classification, linear discriminant analysis for, 296 FT. See Fourier transform Fungal infestations, in citrus fruits, 245–246 Fusarium graminearum, 416 Fusarium verticillioides, 416 Fuzzy classifiers, 166 Fuzzy clustering, 51 Fuzzy evaluation score (FES), 436 Fuzzy logic, 436 Fuzzy outputs, 94 Fuzzy rule base construction, 93–94 Fuzzy sets, 92–93 Fuzzy set theory, 91 G Gabor and wavelet transform (WT), for determining image texture, 150–152 Gabor features (GF), for determining image texture, 152–153 Gabor filter, 69 Galactomyctes citri-aurantii, 246 Gamma rays, 6 Gaussian stationary stochastic processes, 516 Gelatinization characteristics, of cooked rice, 390

572 Index

Global thresholds selection, methods of fuzzy thresholding technique, 44 histogram clustering, 44 isodata algorithm, 43–44 manual selection, 42 minimum error technologies, 44 objective function, 43–44 window extension method, 44 Gluten free breads, 487 Gorda, 290 Gradient-based segmentation, 47–49 operators in, 47–49 Gradient operator, 47 Gradient scales of images, 52 Gradient vector flow (GVF), 49 Grading and sorting, of potatoes advantages, 305 disadvantages to using human inspectors in, 305 machine vision inspection of algorithm design, 314–315 automated defect detection, 310–313 characterization of defects, 314 development, 308–310 vision unit, 313–314 Grain hardness, 404 Grain-scale macro-structure, of cooked grain, 391–393 Graphic algorithm, 52 Grassmann’s laws of additive color mixture, 162 Grating Type I, 177 Gray-level co-occurrence matrices (GLCM), 361 for determining image texture, 150–152 Gray-level histograms (GLH), 360 Gray-level run-length matrices (GLRM), 361 Graypixels, intensity value of, 38, 39, 40, 50, 52 Grayscales of images, 52 Gray values, of monochrome images, 360 Green Isle Foods Company, 427 Green olives, 290, 292 Green spots, of potatoes, 306 Ground-probing radar (GPR), 4 H Haar transform analysis, for texture properties, 513–514 Ham area percentage, 443 Hard-field effect, 6 Hardware-orientated spaces CMYK, 64 RGB, 63–64 YIQ, 63–64 Hazard analysis and critical control points (HACCP), 159 HDFS. See High dimensional feature space Helium pycnometry, 145

Helminthosporium sp., 306 Hidden layers, 83–84 Hierarchical fruit grading method, for apples based on object clustering, 233–234 and fruit classification, 234–237 High dimensional feature space (HDFS), 98 High protease-activity flour (HPAF), on cookie quality parameters, 491 High-speed quality inspection of potatoes (HIQUIP) system, 310 High-temperature drying technology, 487 Histogram, two-dimensional, 44–45 Hot carcass weights, 130 Hough transform algorithm, 511 HSI (hue, saturation, intensity) color spaces, 64 conversion, 163 HSL (hue, saturation, lightness) color spaces, 64 Hue–saturation–value (HSV), 309 Human oriented color spaces HSI, 64–65 HSL, 64 HSV, 64–65 Hunter color parameters, 488–489 Hunter Lab colorimeter, 542 Hunter Lab Universal Software 3.1, 292 Hybrid-based segmentation, 52 Hybrid image-processing system, 115 Hyperspectral imaging, strawberries bruises detection, 275–279 algorithms, 278 Hyperspectral imaging technique, 320, 418 applications optical properties of fruits and juices, 334–335 quality assessment of fruits, 336–344 calibration requirements of instrument, 327–330 determination of absorption and reduced scattering coefficients, 330–333 instrument set-up, 326–327 theory, 324–325 Hyperspectral vision, citrus fruits inspection, 255–256 I Image acquisition designs, 219 devices, 216–217, 220 Image acquisition systems, computer vision, 7 electronics, 11–13 illumination, 8–11 Image-opening and -closing, technique of, 45–46 Image post-processing, 52

Index 573

Image pre-processing, techniques of, 1 contrast enhancing, 39–41 histogram equalization, 40–41 histogram scaling, 40 noise removal, 38–39 linear filter, 38 median filter, 38–39 Image-processing algorithms, 160, 169 Image Pro Plus software 6, 559 Image scales, 52 Image segmentation, 1 of cartilage and bone, 52 classification-based classification methods, 50–51 drawback, 49 feature extraction, 50 gradient-based active contour model (ACM), 48–49 gradient operator, 47 Laplace operator, 47–48 hybrid-based, 52 region-based, 46–47 thresholding-based, 41 image-opening and -closing, 45–46 threshold selection, 42–45 watershed-based, 51–52 Image texture, 1 correlations with tenderness, 151–153 extraction of, 150–151 features, 50 Imaging devices, 5 Imaging spectroscopy. See Hyperspectral imaging Indium gallium arsenide (InGaAs)-based near-infrared (NIR) camera, 5 Inflammatory processes in chicken, 160 Infrared focal plane array (IRFPA), 22 Infrared sensor array, 19 Infrared systems, 17–19 cooled infrared detectors, 19–21 uncooled IR detectors, 21–23 Instrumental spaces, 65–67 Interactance, 366 Interconnected nodes. See Processing elements Internal reflection coefficient, 325 International Commission for Illumination (CIE), 162 Inwhite shrimp (Penaeus setiferus), 191, 196 IR cameras, 17, 19, 22 IR detection, materials used for, 19 IR detectors, semiconductor, 19 J Japanese Beef Marbling Standard, 112 Japanese National Institute of Animal Industry, 113

Jonagold apples, defect segmentation, 227–228 Joule–Thompson gas expansion method, 20–21 K Kahonen-type neural network, 497 Kappa coefficient, 179 Kernel function, 98–99 K nearest-neighbor classifier, 432 Kohonen’s self-organizing map (SOM), 311 Kubelka–Munk equations, 321 function, 365–366 L Lactic fermentation, 290 Lamb, quality evaluation, 129–131 Lambertian Cosine Law, 334 Lamb vision system (LVS), 130 Lamor effect, 26–27 Laplace operator, 47–48 LD. See Longissimus dorsi Learning vector quantization (LVQ) technique, 172 LensEye software, 200 Light absorption, 321 for Golden Delicious apples, 334–335, 337 for Red Haven peaches, 334–335 Light-emitting diode (LED) based instruments, 367 Lighting geometries, 10–11 Lighting system, for defect detection in apples, 218–220 Linear discriminant analysis (LDA), 310 for fruit classification, 296 Linear filter, noise removal using, 38 Line-scan imaging system, 355–356 Lipid distribution, of a brown rice kernel, 387 Liquid-crystal tunable filters (LCTF), 168, 368 Localized boosting classifier, 432 Longissimus dorsi (LD), 52, 112, 113, 114 Longissimus muscle, 119, 120 Luminance features, of melted cheese, 456 LVS. See Lamb vision system Lye, 290 M Machine vision systems assessment of corn quality using breakage, 413 color, 410–411 hardness or vitreousness, 416–417 heat damage, 415

574 Index

Machine vision systems (Contd.) hyperspectral imaging, 418 mold and fungal contamination, 415–416 near-infrared reflectance (NIR) spectroscopy, 418 real-time husk deduction measurements, 418 seed viability, 417–418 separation of shelled corn from residues, 418 size and shape, 412–413 stress cracks, 414 synchrotron Fourier transform infrared (FTIR) microspectroscopy, 418 color measurement using, 196 computer-aided, 379 inspection system with, 160, 163 olives, 289 measurement of seed characteristics using color property, 406 concept of simple counting, 401 integration and automation of analysis, 406–408 internal seed characteristics, 403–404 quality characteristics, 404–406 sample variability and, 406 whole seed analysis, 402–403 olive sorting, application in image analysis system, 295 image processing, 295–296 industrial system, 294–295 Potatoes (Solanum tuberosum) algorithm design, 314–315 automated defect detection, 310–313 characterization of defects, 314 development, 308–310 vision unit, 313–314 poultry industry, applications in, 158 wheat quality evaluation of area-scan imaging, 354–355 development of separation algorithms, 358–359 line-scan imaging, 355–356 morphological, color, and textural algorithms, 359–362 overview, 353–354 practical applications, 370–371 sample presentation devices, 356–358 Magness-Taylor (MT) probe, 337 Magnetic resonance imaging (MRI) of citrus fruits, 256–257 scanner, 27 Mahalanobis algorithm, 300–301 application of, 299 Mahalanobis distance, 226, 298, 301 classification, 178 definition, 501

Mahi mahi, 198–199 Maillard browning reaction, 451, 487, 488, 492, 551 Manzanilla olives, 290, 292 Marker-controlled watershed, 146 MATROXiTOOLS, 503 Matrox Meteor RGB frame grabber, 295 Maturity index (MI), 244 Meat cuts quality evaluation, future perspectives of, 131 Meats. See Cooked meats Median filter, noise removal using, 38 Melanosis, 191, 197 Melting quality, of cheese, 448–451 Membership function, 93 Mercury porosimetry, 145 Michelson interferometer, 367 Microbolometers, 22, 23 Microbridge structure, 23 Microcrystalline cellulose, use in cheese, 455 Miniscan Hunter Lab spectrocolorimeter, 292 Minolta colorimeter meter, 498 Minutemaid orange juice, absorption and scattering coefficient of, 335 MLR. See Multiple linear regression Model-based methods autoregressive model, 71 fractal model, 70–71 Moisture tomograms, 30 Mold and fungal contamination, of seeds, 415–416 Morphological features, of an object, 360 Morphological image processing, 51 Morphological image-processing algorithm, based on watershed segmentation, 359 Mozzarella cheeses, 450 Muffins, 495–496, 498–499 Multilayer back-propagated neural network method, for defect detection in apples, 236–237 Multilayer feed-forward neural network (MLFN-NN), 309, 314–315 Multiple linear regression (MLR), 122, 125 Multispectral image acquisition system, 168–169 Multispectral imaging system, 163, 164, 165–166 in poultry industry applications, 158 Multispectral imaging technique, 50 Muscles, fat area ratio, 113 N Nearest neighbor distance calculation, 90 distance metrices, 90 Near infra-red hyperspectral imaging, 367–369 Near-infrared (NIR) imaging, for bruises detection in strawberries, 275

Index 575

Near-infrared reflectance (NIR) spectroscopy, 418 measurement modes, 365–366 Neighboring dependence matrix (NDM), 68 Neural networks grading system for fruit and vegetables, 296 olive-grading process, with a hidden layer for, 298 quality inspection in citrus fruits, 251, 254 unsupervised, 51 Neuro-fuzzy system, 161 Noise removal linear filter, 38 median filter, 38–39 Non-black bodies, 18–19 Non-dispersive systems, 367 Non-enzymatic Maillard browning reaction, 483, 485 Non-iterative thinning methods, 457 Non-linear discriminant analysis, for classification of peaches, 296 Non-parametric classifiers, of pizza, 432 Non-stationary radiation transfer theory, 323 Noodles developments and further applications in analysis artificial neural networks, use of, 541–542 flat-bed scanners, use of, 539-540 impact of addition of functional food additives, 540-541 factors influencing quality of, 527-529 imaging quality assessment of flour quality, 529-530 initial research, 530-535 measuring impacts of external grading factors frost damage, 538 Fusarium head blight (FHB), 537-538 sprout damage, 536-537 production overview of, 524-527 Normalization, 298 Nuclear magnetic resonance imaging (NMR imaging), 307–308, 382 O Oat kernel area and kernel mass, 404 Objective function, 51 method, for threshold selection entropy-based, 43–44 variance-based, 43–44 Object measurement methods color hardware-orientated spaces, 63–64 human-orientated spaces, 64–65 instrumental spaces, 65–67

shape, 59 size-dependent measurements (SDM), 60 size-independent measurements (SIM), 60–63 size, 58 drawbacks of, 59 Oiling-off, in cheese, 453–454 Olea europea arolensis, 290 Olea europea pomiformis, 289, 290 Olives. See Table olives Oncorhynchus gorbuscha, 200 On-line vision system, 160 Optical properties of fruits and juices, 334–335 techniques for measuring. See also Hyperspectral imaging technique absorption and scattering, 321 forms of light interaction, 320–321 frequency-domain techniques, 322–323 spatially-resolved spectroscopic/imaging techniques, 323–324 time-resolved techniques, 321–322 Optimas, 42 Oreochromis niloticus, 205 Oryza sativa L. See Rice P Parallelepiped classification, 178 Partial least squares (PLS), 122, 124, 125 discriminant analysis, 299, 301 multivariant, for olive-grading process, 298 for olive classification, 298 Partial least squares (PLS) discriminant technique (PLS-DA), 298 Partial least squares regression (PLSR) technique, for determining image texture, 152 Passive mode of sensors, 11 PCR. See Principal component regression Peak intensity values, 329 Peeled olives, 290, 293 Peltier–Thompson effect, 21 Penaeus setiferus, 191, 196 Penicillium digitatum, 245 Penicillium sp., 416 PEs. See Processing elements Photodiode array spectrophotometer, 161 Photoluminescence imaging technique, 390 Photoshop, 42 Pink nitrosyl myochromogen, 149 Pink salmon (Oncorhynchus gorbuscha), 200 Pixel-oriented techniques, 251, 252 Pixel(s) intensity value of, 38, 39, 40, 50, 52 run, 122 Pizza crust, 428

576 Index

Pizza making application of toppings color evaluation, 438–441 topping percentage and distribution, evaluation, 441–443 base production classification, 431–433 feature extraction, 428–429, 429–431, 431 sauce spread classification, 435–437 color feature extraction, 434–435 Planck’s equation, 6, 27 Planck’s law, 18 PLS. See Partial least squares Pores and porosity of cooked meats, 145 correlation with water content, processing time, and texture, 147–149 measurement of, 146–147 Pork color assessment, 125–126 marbling assessment, 126 quality evaluation carcass composition, prediction, 129 carcass grades, prediction, 126–129 color and marbling attributes, characterization, 125–126 Potato chips and French fries computer vision system analysis classification, 549 feature extraction, 548–549 image acquisition, 547 image pre-processing and segmentation, 548 image analysis applications Browning evaluation, 556–563 sorting, 550–555 Potatoes (Solanum tuberosum), 305 classification, 307–308 machine vision inspection of algorithm design, 314–315 automated defect detection, 310–313 characterization of defects, 314 development, 308–310 vision unit, 313–314 surface defects, 306 Poultry carcasses color-image-processing procedures for, 163 defect recognition in, 163 heart disease, detection of, 167–169 quality inspection by dual-band spectral imaging, 170–171 spectral characterization of, 163–165 systemic disease, detection of, 166–167 quality inspection, 165 condemnation during, 157

and grading of, 158–159 Poultry image classifications airsac classification, 171–172 quality classification by texture analysis fast power spectra of spectral images, 173 fractal analysis, 173–176 neural network models, 176–177 spectral poultry image classification in frequency domain, 172–173 supervised algorithms for hyperspectral image classification accuracy of classifiers for contaminant identification, 181–182 classification methods, 178–179 comparison of classification methods, 180–181 hyperspectral image characteristics for classification, 179–180 hyperspectral imaging system, 177–178 Principal component analysis (PCA), 50–51, 61, 62, 166, 435 for olive classification, 298 Principal component regression (PCR), 122, 124 Principal components, 298 Processing elements (PEs), 82–84 Protein denaturation, 146 Prototypes algorithms, 91 Pyroelectricity, 21–23 Pyroelectric tube, 22 first-generation, 22 Q Quality assurance, using machine vision, 3 Quality inspection, color imaging for characterizing wholesomeness, 161–162 splenomegaly detection, 159–160 viscera inspection, 160–161 R R. dominica, 364, 368 Radial basis function network classifier, 432 Radiant intensity, 64 Radiowaves (RF), 6 pulse, for citrus fruits, 257 Rapid acquisition, of spatially-resolved scattering profiles, 324 Rapid acquisition relaxation enhanced (RARE) imaging method, 393 Readout integrated circuit (ROIC), 23 Readout noise, 38 Red (R), Green (G), and Blue (B) color model, 547, 552 Reduced scattering coefficient, 325 Region-based method, 60 applications of, 61

Index 577

Region-based segmentation, techniques of growing-and-merging (GM), 46 splitting-and-merging (SM), 46–47 Region-oriented segmentation techniques, in citrus fruits, 252–253 Regions of interests (ROIs), 166, 202 class, 178 Relative refractive index, 325 Remote probing, 24 Remote sensing, 24 Resilient back-propagation (Rprop), 298–299 RGB (red, green, blue), 63–64 color space, 435 Rhizoctonia solani, 306 Ribeye muscle, 113 Rice (Oryza sativa L.), quality evaluation of cooked rice cell-scale micro-structure, 393–397 grain-scale macro-structure, 391–393 water distribution, 390–391 quality parameters for rice, 377–378 raw rice compound contents and distribution, 383–390 physical properties, 379–381 water content and distribution, 381–382 Rich Tea type biscuits, 493 Ring illuminator, 10–11 Ripeness grading, in strawberries, 272–273 Rovi-Tech’s ILB-25, 563 Run length matrix (RLM) method, 68 for determining image texture, 150–152 S S. cerealella, 368 S. oryzae, 364, 368 Salt-injected cheeses, 466 San Fuji apples, defect segmentation, 228 SAS stepwise procedure, 122, 124 Sauce area percentage, 435 SC. See Semispinalis capitis; Statistical classification Scanning electron microscopy (SEM), 393 Scattering, of light, 321 for Golden Delicious apples, 334–335, 337 for Red Haven peaches, 334–335 Scene lighting, for citrus fruits, 248–249 Schottky barrier diode, 22 Schreiber test, of meltability, 448 SD. See Semispinalis dorsi SDM. See Size-dependent measurements Seafood, visual quality of color, 195 color evaluation of carbon-monoxide treated seafood, 197–200

combining color with other quality parameters, 205–206 MV and colorimeter evaluation of sturgeon color, comparison of, 204–205 shrimp color, 196–197 sorting whole salmon by skin color, 200–204 space, 196 shape evaluation of rigor mortis in sturgeon, 194–195 overview, 193–194 size determination of shrimp weight, count, and uniformity ratio, 190 overview, 189–190 oyster volume, 191–193 Seed viability and vigor, 417–418 Segmentation. See Image segmentation Selector 4000 from Multiscan Technologies, 301 Self-organizing map (SOM), 51, 497 Semiconductor IR detectors, 19 Semispinalis capitis (SC), 113 Semispinalis dorsi (SD), 113 Sensing modalities, 29 Sensor operation, 11 Sensory scores, 115 Separation algorithms, 358–359 Septicemia, 157 Sequential forward selection (SFS) method, 553 Shape grading in apples, 222 in strawberries, 268–270 Shape measurement applications of, 59 size-dependent measurements (SDM), 60 size-independent measurements (SIM) boundary-based method, 61–63 region-based method, 60–61 11 SHARC Digital Signal Processors, 310, 314 Shredded cheese thinning methodologies and skeletonization algorithm, 457–458 X–Y sweep measurements, 458–462 Shrinkage of cooked beef joints, 62 Shrinkage of cooked meats, 143 correlations with yield, water content, and texture, 143–145 size and shape measurement average diameter, short axis, long axis, and perimeter, 140–141 surface area and volume, 141–143 SIM. See Size-independent measurements Simpson’s method, for oyster volume estimation, 193 Single-term linear regression (STLR), 162

578 Index

Size-dependent measurements (SDM), 60 Size grading in strawberries, 268, 270 Size-independent measurements (SIM), methods of, 60–63 Size measurement drawback of, 59 Feret’s diameter, 58 major axis, 58 minor axis, 58 Skeletonization algorithms, 457 Skin damage in olives, 292 Smooth-rough crossover (SRC), 555 Snake algorithm, quality inspection of apples, 229–230 Snake photons, 321 Snakes. See Active contour model Soft-field effect, 6 Soft olives, 292 Softwares Hunter Lab Universal Software 3.1, 292 Image Pro Plus software 6, 559 LensEye, 200 Soft X-ray imaging, 362–364 Solanum tuberosum. See Potatoes Sorting, of olives cameras in industrial system of, 294 machine vision systems, application of image analysis system, 295 image processing, 295–296 industrial system, 294–295 process Box–whisker diagrams, 297 flowchart of, 295 Sorting, of potato chips color classification techniques, 553–554 and frying temperature, 554 models, 552–553 texture, 554–555 Spatially-resolved spectroscopic/imaging techniques, 323–324 Spatial moment. See Region-based method Spectral angle mapper (SAM), 178 Spectral calibration, 327 Spectral energy distribution, 9 Spectral imaging method, 160, 167 Spectral power distribution (SPD), 162 Spectrophotometer, 162 Specular reflectance, 321 Spin–lattice excitation, 27 Spin–spin relaxation, 27 Splenomegaly, 160 SRM. See Structural risk minimization Statistical classification, types Bayesian classification, 87–88

discriminant analysis, 88–89 nearest neighbor, 89–91 Statistical methods, for texture measurements co-occurrence matrix, 67–68 neighboring dependence matrix, 68 run-length matrix, 68 Statistical projection methods, 298 Storage-frame array, 12 Strawberries (Fragaria x ananassa Duch.) characteristics, 265–266 production, 265, 266 quality evaluation in Japan, 265–267. See also Strawberries, quality inspection varieties, 265 Strawberries, quality inspection Akihime strawberries, 270 anthocyanin distribution estimation, 283–284 belt-type strawberry sorting system, 271 bruises detection, 273–274 color-image capturing system, 274 color imaging for, 274–275 hyperspectral imaging, 275–279 near-infrared (NIR) imaging, 275 challenges, 284–285 estimation of firmness and soluble-solids content firmness measurement, 280–283 internal quality measurement, 279–280 fecal contamination, detection of, 279 flowchart of grading program for on-line system, 272 grading of size, shape, and ripeness, 267–273 quality grades standards, 267–268 ripeness grading, 272–273 size and shape judgment study, 268–272 soluble-solids content measurement, 283 Streptomyces sp., 306 Stress cracks, 414 Stress-fissure detection, 381 Structural methods, for texture measurements, 68 Structural risk minimization (SRM), 97 Sugar and dough properties, 494 Supervision learning, 49 Support vector machine (SVM), 437 classifiers, performance of, 100 multi-classification, 99–102 SVM. See Support vector machine Synchrotron Fourier transform infrared (FTIR) microspectroscopy, 418 Systemic problems in chicken, 165 T T. castaneum larvae, 364 Table olive(s), 289 brine, 291

Index 579

classes of, 291–292 classification algorithms, 296–299 classification by quality, 291 characterization of defects, 292–293 types of defects, 292 classification of artificial neural networks (ANNs) for, 298–299 principal component analysis (PCA) for, 298 fermentation of, 290–291 flesh damage in, 292 green, 290, 292 industrial applications, 300 machine vision for inspection of, 289 Manzanilla, 290 peeled, 290, 293 production process, 290–291 flowchart of, 290 real-time sorting system, 299–300 skin damage in, 292 soft, 292 wired, 290, 293 Table olive-grading process, 298 Table olive-harvesting season, 294 Table-olive producers, industrial needs of, 293–294 Table olives, sorting of cameras in industrial system of, 294 machine vision systems, application of image analysis system, 295 image processing, 295–296 industrial system, 294–295 process Box–whisker diagrams, 297 flowchart of, 295 Table olive-sorting machine, 301 Textural attributes, of chips, 494 Textural defects in olives, 292 Textural features, of an object, 360–361 Textural properties, of bakery products, 512–517 Texture measurements, methods of model-based methods autoregressive model, 71 fractal model, 70–71 statistical methods co-occurrence matrix, 67–68 neighboring dependence matrix, 68 run-length matrix, 68 structural methods, 68 transform-based methods, 68 convolution mask, 69 Fourier transform, 69 wavelet transform (WT), 69–70 Thermal imaging, 17 systems, 369–370

Thermoelectrics, 22 Thermographic cameras, 19 Thermographic photography, 17 Three-dimensional plotting model, of grain, 385 Thresholding-based segmentation, types of, 41 Thresholds selection. See Global thresholds selection, methods of types of, 41 Tilapia (Oreochromis niloticus), 205 Tiles, 146 Time-resolved techniques, 321–322 Tomograms, moisture, 30 Tomographic imaging, 23 electrical tomography (ET), 29–31 geometries of, 24 nuclear tomography, 24–25 computed tomography (CT), 25–26 magnetic resonance imaging (MRI), 26–28 Topping area percentage, 443 Total processing time (TPT), 148 Transfer functions, classification, 83 Transflectance, 365 Transform-based methods, for texture measurement, 68 convolution mask (CM), 69 wavelet transform (WT), 69–70 Transform function, 40 Transmittance, 365 Transport albedo, 325 Trapezius muscle, 120 Trichoderma viride, 416 Tri-chromatic theory, 63 Tristimulus values, 162 Tuber diseases, in potatoes, 306–307 germination, 306 Tuna, 197 Two-dimensional histogram, 44–45 U Ultrasonic measuring system, 14 Ultrasound for food quality and evaluation ultrasound images, drawbacks of, 16 uses, 13–14 velocity measurement, 14–16 USDA standards beef grading system, 112, 116 for body and texture of shredded cheese, 456 for grades of potatoes, 307 marbling scorecard, 115 quality grades, 119, 122 yield grading scales, 119 US RS170 video standards, 13

580 Index

V Vapnik Chervonenkis dimension (VCD), 97 VCD. See Vapnik Chervonenkis dimension Visible and near-infrared (Vis/NIR) spectrophotometer, 167 Visible and near-infrared (Vis/NIR) spectroscopic analysis, 169 Visible and near-infrared (Vis/NIR) spectroscopic technologies, 161–162 Visible and near-infrared (Vis/NIR) spectroscopy, 320, 337, 339 W Warner-Bratzler shear (WBS) force measurement, 151 Waterobjective function, 51 Watershed algorithm, 146 Watersheds-based segmentation technique, 51–52 Wavelet features (WF), for determining image texture, 152–153 Wavelet Gabor features (WGF), for determining image texture, 152–153 Wavelet transform (WT), 69–70 Wave motion, 23 Wheat global production, 351 grading factors, 351–352 Wheat, quality evaluation of machine-vision based inspection area-scan imaging, 354–355 development of separation algorithms, 358–359

line-scan imaging, 355–356 morphological, color, and textural algorithms, 359–362 overview, 353–354 practical applications, 370–371 sample presentation devices, 356–358 near infra-red hyperspectral imaging, 367–369 near infra-red (NIR) spectroscopy instrumentation, 366–367 measurement modes, 365–366 soft X-ray imaging, 362–364 thermal imaging, 369–370 Wheaton shucking machine, 190 White fish (Chirostoma estor estor), 190 White shrimp (Penaeus setiferus), 191 Wholesome Poultry Products Act, 158–159 Wilks’ lambda analysis, 501 Wired olives, 290, 293 Wiring noise, 38 X XC-003 Sony color 3-CCD camera, 295 Xenon ionization chambers, 25 X-rays, 6 source, 25 Xylene-soaking process, 388 X-Y sweep measurements, of shredded cheese, 458–462 Z Zinc sulfide, 20

Food science and technology International Series

Maynard A. Amerine, Rose Marie Pangborn, and Edward B. Roessler, Principles of Sensory Evaluation of Food. 1965. Martin Glicksman, Gum Technology in the Food Industry. 1970. Maynard A. Joslyn, Methods in Food Analysis, second edition. 1970. C. R. Stumbo, Thermobacteriology in Food Processing, second edition. 1973. Aaron M. Altschul (ed.), New Protein Foods: Volume 1, Technology, Part A—1974. Volume 2, Technology, Part B—1976. Volume 3, Animal Protein Supplies, Part A—1978. Volume 4, Animal Protein Supplies, Part B—1981. Volume 5, Seed Storage Proteins—1985. S. A. Goldblith, L. Rey, and W. W. Rothmayr, Freeze Drying and Advanced Food Technology. 1975. R. B. Duckworth (ed.), Water Relations of Food. 1975. John A. Troller and J. H. B. Christian, Water Activity and Food. 1978. A. E. Bender, Food Processing and Nutrition. 1978. D. R. Osborne and P. Voogt, The Analysis of Nutrients in Foods. 1978. Marcel Loncin and R. L. Merson, Food Engineering: Principles and Selected Applications. 1979. J. G. Vaughan (ed.), Food Microscopy. 1979. J. R. A. Pollock (ed.), Brewing Science, Volume 1—1979. Volume 2—1980. Volume 3—1987. J. Christopher Bauernfeind (ed.), Carotenoids as Colorants and Vitamin A Precursors: Technological and Nutritional Applications. 1981. Pericles Markakis (ed.), Anthocyanins as Food Colors. 1982. George F. Stewart and Maynard A. Amerine (eds.), Introduction to Food Science and Technology, second edition. 1982. Malcolm C. Bourne, Food Texture and Viscosity: Concept and Measurement. 1982. Hector A. Iglesias and Jorge Chirife, Handbook of Food Isotherms: Water Sorption Parameters for Food and Food Components. 1982. Colin Dennis (ed.), Post-Harvest Pathology of Fruits and Vegetables. 1983. P. J. Barnes (ed.), Lipids in Cereal Technology. 1983. David Pimentel and Carl W. Hall (eds.), Food and Energy Resources. 1984. Joe M. Regenstein and Carrie E. Regenstein, Food Protein Chemistry: An Introduction for Food Scientists. 1984. Maximo C. Gacula, Jr., and Jagbir Singh, Statistical Methods in Food and Consumer Research. 1984. Fergus M. Clydesdale and Kathryn L. Wiemer (eds.), Iron Fortification of Foods. 1985. Robert V. Decareau, Microwaves in the Food Processing Industry. 1985.

582 Food Science and Technology: International Series

S. M. Herschdoerfer (ed.), Quality Control in the Food Industry, second edition. Volume 1— 1985. Volume 2—1985. Volume 3—1986. Volume 4—1987. F. E. Cunningham and N. A. Cox (eds.), Microbiology of Poultry Meat Products. 1987. Walter M. Urbain, Food Irradiation. 1986. Peter J. Bechtel, Muscle as Food. 1986. H. W.-S. Chan, Autoxidation of Unsaturated Lipids. 1986. Chester O. McCorkle, Jr., Economics of Food Processing in the United States. 1987. Jethro Japtiani, Harvey T. Chan, Jr., and William S. Sakai, Tropical Fruit Processing. 1987. J. Solms, D. A. Booth, R. M. Dangborn, and O. Raunhardt, Food Acceptance and Nutrition. 1987. R. Macrae, HPLC in Food Analysis, second edition. 1988. A. M. Pearson and R. B. Young, Muscle and Meat Biochemistry. 1989. Marjorie P. Penfield and Ada Marie Campbell, Experimental Food Science, third edition. 1990. Leroy C. Blankenship, Colonization Control of Human Bacterial Enteropathogens in Poultry. 1991. Yeshajahu Pomeranz, Functional Properties of Food Components, second edition. 1991. Reginald H. Walter, The Chemistry and Technology of Pectin. 1991. Herbert Stone and Joel L. Sidel, Sensory Evaluation Practices, second edition. 1993. Robert L. Shewfelt and Stanley E. Prussia, Postharvest Handling: A Systems Approach. 1993. R. Paul Singh and Dennis R. Heldman, Introduction to Food Engineering, second edition. 1993. Tilak Nagodawithana and Gerald Reed, Enzymes in Food Processing, third edition. 1993. Dallas G. Hoover and Larry R. Steenson, Bacteriocins. 1993. Takayaki Shibamoto and Leonard Bjeldanes, Introduction to Food Toxicology. 1993. John A. Troller, Sanitation in Food Processing, second edition. 1993. Ronald S. Jackson, Wine Science: Principles and Applications. 1994. Harold D. Hafs and Robert G. Zimbelman, Low-fat Meats. 1994. Lance G. Phillips, Dana M. Whitehead, and John Kinsella, Structure-Function Properties of Food Proteins. 1994. Robert G. Jensen, Handbook of Milk Composition. 1995. Yrjö H. Roos, Phase Transitions in Foods. 1995. Reginald H. Walter, Polysaccharide Dispersions. 1997. Gustavo V. Barbosa-Cánovas, M. Marcela Góngora-Nieto, Usha R. Pothakamury, and Barry G. Swanson, Preservation of Foods with Pulsed Electric Fields. 1999. Ronald S. Jackson, Wine Science: Principles, Practice, Perception, second edition. 2000. R. Paul Singh and Dennis R. Heldman, Introduction to Food Engineering, third edition. 2001. Ronald S. Jackson, Wine Tasting: A Professional Handbook. 2002. Malcolm C. Bourne, Food Texture and Viscosity: Concept and Measurement, second edition. 2002. Benjamin Caballero and Barry M. Popkin (eds), The Nutrition Transition: Diet and Disease in the Developing World. 2002. Dean O. Cliver and Hans P. Riemann (eds), Foodborne Diseases, second edition. 2002. Martin Kohlmeier, Nutrient Metabolism, 2003. Herbert Stone and Joel L. Sidel, Sensory Evaluation Practices, third edition. 2004. Jung H. Han, Innovations in Food Packaging. 2005. Da-Wen Sun, Emerging Technologies for Food Processing. 2005. Hans Riemann and Dean Cliver (eds) Foodborne Infections and Intoxications, third edition. 2006.

Food Science and Technology: International Series 583

Ioannis S. Arvanitoyannis, Waste Management for the Food Industries. 2008. Ronald S. Jackson, Wine Science: Principles and Applications, third edition. 2008. Da-Wen Sun, Computer Vision Technology for Food Quality Evaluation. 2008. Kenneth David, What Can Nanotechnology Learn From Biotechnology? 2008. Elke Arendt, Gluten-Free Cereal Products and Beverages. 2008.

Computer Vision Technology for Food Quality Evaluation

Empirical Evaluation Methods in Computer Vision

Empirical Evaluation Methods in Computer Vision

Nondestructive Food Evaluation: Techniques to Anyaluze Properties and Quality (Food Science and Technology)

Sensory Evaluation of Food (Food Science and Technology)

Computer Vision

Computer Vision

Nondestructive Evaluation of Food Quality: Theory and Practice

Fats in Food Technology (Sheffield Food Technology)

Statistical quality control for the food industry

Statistical Quality Control for the Food Industry

Case Studies in Food Microbiology for Food Safety and Quality

Automation for Food Engineering: Food Quality Quantization and Process Control

Automation for food engineering: food quality quantization and process control

Automation for Food Engineering: Food Quality Quantization and Process Control

An evaluation platform for semantic web technology

Real-time Vision for Human-computer Interaction

Image processing for computer graphics and vision

Computer vision and applications

Embedded computer vision

Fundamentals of Computer Vision

Computer Vision for Human-Machine Interaction

Real-Time Vision for Human-Computer Interaction

Real-Time Vision for Human-Computer Interaction

Introductory techniques for 3-D computer vision

Computer vision and applications

Embedded Computer Vision

Sensory Evaluation Practices, Third Edition (Food Science and Technology)

Quality Management for the Technology Sector

Frying - Improving Quality (Woodhead Publishing in Food Science and Technology)

Waste Management for the Food Industries (Food Science and Technology) (Food Science and Technology)

Computer Vision Technology for Food Quality Evaluation

Empirical Evaluation Methods in Computer Vision

Empirical Evaluation Methods in Computer Vision

Nondestructive Food Evaluation: Techniques to Anyaluze Properties and Quality (Food Science and Technology)

Sensory Evaluation of Food (Food Science and Technology)

Computer Vision

Computer Vision

Nondestructive Evaluation of Food Quality: Theory and Practice

Fats in Food Technology (Sheffield Food Technology)

Statistical quality control for the food industry

Statistical Quality Control for the Food Industry

Case Studies in Food Microbiology for Food Safety and Quality

Automation for Food Engineering: Food Quality Quantization and Process Control

Automation for food engineering: food quality quantization and process control

Automation for Food Engineering: Food Quality Quantization and Process Control

An evaluation platform for semantic web technology

Real-time Vision for Human-computer Interaction

Image processing for computer graphics and vision

Computer vision and applications

Embedded computer vision

Fundamentals of Computer Vision

Computer Vision for Human-Machine Interaction

Real-Time Vision for Human-Computer Interaction

Real-Time Vision for Human-Computer Interaction

Introductory techniques for 3-D computer vision

Computer vision and applications

Embedded Computer Vision

Sensory Evaluation Practices, Third Edition (Food Science and Technology)

Quality Management for the Technology Sector

Frying - Improving Quality (Woodhead Publishing in Food Science and Technology)

Waste Management for the Food Industries (Food Science and Technology) (Food Science and Technology)

Recommend Documents