Understanding And Applying Machine Vision Part 3 potx

Page 61TABLE 4.6 Machine Vision, Robotic Related Manipulation of Separated Workplaces on Conveyors Bin Picking Manipulation of Manufacturing Process Finishing, sealing,deburring, cutti

Figure 4.23Early system from Gould Electronics providing two-dimensional vision to

guide robot for assembly operation

Figure 4.24Early system from Machine Vision International provides correction tosix degrees of freedom associated with position of car as it installs

windshield

Page 61TABLE 4.6 Machine Vision, Robotic Related

Manipulation of Separated

Workplaces on Conveyors

Bin Picking Manipulation of

Manufacturing Process

Finishing, sealing,deburring,

cutting, processcontrol, flashremoval, liquidgasketing

In-processInspectionFastening, spotwelding,riveting, arcWelding, bolting,screwing,

nailing, gluing,staplingFitting, partspresentationMating of parts

Several analyses of applications that have been conducted suggest that approximately 42% of the applications relate to inspection (gaging, cosmetic, and verification), 45% to visual servoing location analysis of which robot guidance is only one application, and 13% to part identification and recognition Significantly, robot vision applications often require systems capable of inspection in addition to guidance

4.4—

Overview of Generic Machine Vision Benefits and Justification

The opportunities for machine vision are largely in inspection and assembly operations Even in the latter case, many

of the applications will involve inspection (e.g., of tasks), verification, flaw detection, and so on In conjunction with such tasks, people are only 70–85% effective, especially when dealing with repetitive tasks According to researchers

at the University of Iowa, people asked to perform the visual sorting task of picking out a minority of black Ping-Pong balls from a production line of white ones allowed 15% of the black balls to escape Even two operators together were only about 95% effective

Page 62People have a limited attention span, which makes them susceptible to distractions People are also inconsistent

Individuals themselves often exhibit different sensitivities during the course of a day or from day to day Similarly, there are inconsistencies from person to person, from shift to shift, and so on The eye's response may also be a

performance limiter

However, people offer some advantages over machine vision People are more flexible and can be trained for many tasks People can make adjustments to compensate for certain conditions that should be ignored For example, a label inspection system would have to be tolerant of the range of blue saturation that may be permissible A person can accept anything between pastel yellow and virtually orange if that much of a variance is acceptable On the other hand,

to be tolerant of such a variance, a machine vision system may require its threshold sensitivity be set such that it then accepts labels that are torn People are also quite capable of interpreting the true nature of a condition and, when

trained, can take routine action to correct for a pending process failure

The justification for machine vision need not be based solely on labor displacement A 1984 Booz-Allen Hamilton study (Duncan and Bowen) cited two elements in the cost of quality: the cost of control and the cost of failure The essence of the study suggests that one must consider the savings stemming from the cost of failure in any justification equation The cost of control is generally easy to quantify and includes the prevention and appraisal measures

employed in a factory to find defects before products are shipped to customer-inspection and quality control labor costs and inspection equipment

The cost of failure is much more difficult to quantify and includes internal failures resulting in materials scrap and rework and external failures that result in warranty claims, liability, and recall orders as well as the hidden costs (e.g., the loss of customers)

Machine vision should be considered wherever the prevention of failure or the reduction of the cost of failure is a priority, which should be throughout manufacturing industries Machine vision can be the primary means to avoid internal and external failures

For example, use of a machine vision system in a manufacturing process can avoid the production of scrap Unlike a human inspector who will only detect a reject condition, a machine vision system can spot trends, - trends indicative of incipient conditions that will lead to the production of scrap

Laser gauges, as well as linear array sensors, are available that can make measurements right on or immediately after a machine tool The dimensional or surface finish data gathered by such systems are used as a guide for readjusting the machine tool or replacing the cutting tool before the machine produces scrap

Many industries have jumped on the statistical process control (SPC) philosophy bandwagon Trend analysis,

frequency distribution, and histogram formats for each of the sensors in a system are used to interpret data and report changes in production quality levels In many such cases, this kind of data is

Page 63available only the first time from systems that perform 100% inspection Assessment of the data and its interpretation

in the light of corrective action to take to prevent out-of-specification conditions is a process made possible because of the machine vision equipment Both process control and quality control are possible with machine vision systems.Significantly, avoiding deviations in quality can impact on downstream operations such as assembly By guaranteeing that every piece is in an acceptable condition, one can avoid schedule upsets or the need to reschedule an operation because only defective parts are available Among others, the result of process monitoring and trend analysis could be increased machine uptime or improved capital productivity, that is, increased production capacity without additional equipment and associated floor space

Despite the amount of data now available for processing, an ancillary benefit is reduced paperwork since record

keeping is automated Data transfer between a hierarchy of controllers and computers is easily possible

In those cases where rejects are not prevented, machine vision system can possibly be used to detect conditions before value is added A good example of this can be found in the electronics industry (Figure 4.25) It has been estimated that a fault found on a bare printed circuit board immediately after fabrication only costs $0.25 to repair Once the board is fully loaded with components, the cost to repair that same bare-board reject condition is estimated at $40 before installation in a piece of equipment or shipment As can be appreciated, the costs become commensurately higher to effect that same repair with each value-adding step in manufacturing

Figure 4.25Printed circuit board inspection in electronics industry offers manyopportunities for detection of reject conditions before adding value

(courtesy of Teradyne/Control Automation)

Page 64Similarly, where rejects are not preventable, separating scrap into that which can be reclaimed from that which cannot

is possible with machine vision In the case of thick-film circuits, for example, the detection of the reject before firing permits the reuse of the substrate In the case of machined parts, parts that have dimensions that exceed the maximum tolerance limit can generally be reworked, while those that exceed the minimum tolerances cannot Machine vision systems designed to make measurements on parts can be used to make the distinction, both on-line, as with the

previously mentioned laser gage types, and off-line, with television optical comparator analogs

Real-time machine vision techniques can flag conditions and indicate the need for corrective action before a process goes out of specification or at the very least after only a few rejects are experienced Significant reductions in scrap and rework costs can be achieved from the consistency of flexible automation such as machine vision (Figure 4.26)

Figure 4.26Inex Vision Systems/BWI monitors positioning of

labels at rate of 1200 per minute

Page 65

Figure 4.27Early RVSI/Automatic vision system for weld seamguidance to relax requirements of fixturing.

Clearly, machine vision has value although the tangible costs of scrap and rework are often hidden in manufacturing overheads, thus making it difficult to expose the true cost associated with producing a bad product; for example, a percentage of work-in-process inventory might be held pending scrap or rework decisions Rework, similar to

inventory, is subject to shrinkage and to annual carrying costs Unfortunately, it is difficult to quantify the savings that result from making the product right the first time

Another area to investigate that represents an opportunity for machine vision is one where expensive hard tooling is required to hold a part for an operation (Figure 4.27) This may be avoidable totally or is at least a cheaper flexible fixturing substituted if a machine vision system is used In this case the system can

Page 66provide location analysis, that is, so-called software fixturing A key to this requirement is where setup time is lengthy and the amount of time a part is actually being operated on is very small relative to the total cycle time associated with

an operation Significantly, machine vision may offer increased flexibility, especially in assembly operations, that is, flexibility to produce a wider variety of parts with shorter lead times, better response times to changes in designs, and

so on

Another area for the use of machine vision is where a high incidence of equipment breakdown (Figure 4.28) is

experienced because of such problems as over- and under-size or misshapened and/or warped parts or misoriented parts A machine vision system upstream of the feeder mechanism can reduce or even eliminate downtime

A situation that definitely warrants a machine vision system is one that involves the inventory of parts because

inspection may result in the rejection of a complete batch based on statistical sampling techniques A 100% inspection assures every part is good so ''just-in-time" inventory can be a by-product, with a corresponding reduction in the

material handling and damage experienced by handling Similarly, machine vision opportunities exist where inspection

is a production bottleneck

Figure 4.28Early Opcon system verifies that label is properly positioned Labels appliedinadvertently to area where flash exists gum up blades of deflashing unit,

resulting in equipment downtime

Page 67

As with the justification for robots, one can look for applications related to unhealthy or hazardous environments The Occupational Safety and Health Administration (OSHA) has expressed concerns about the operator's well being: the noise level is too high, the temperature is too hot, the products are too heavy, and so on It may be that the environment includes contaminants (metal dust or vapors) that can be injurious to a person The converse may also be a

justification People may bring contaminants into the environment that can damage the product, for example, dust, causing damage to polished surfaces

Where operation experiences errors due to operator judgment, fatigue, inattentiveness, or oversight brought about because of the dullness of the job, machine vision opportunities exist (Figures 4.29, 4.30, 4.31 and 4.32) Certainly, when an operation is experiencing a capital expansion mode, machine vision should be considered in lieu of

alternative, less effective, more costly methods Where automation is contemplated as a substitute for people, it should

be understood that as people are removed, so are their senses, especially sight When contemplating automation, an analysis is necessary to assure that loss of sight will not affect the production process

Figure 4.29ORS Automation system inspects magnetic and signature strips ofcredit cards for blemishes at rate of 400 per minute.

Page 68

Figure 4.30Early system from Vanzetti Vision Systems performs "stranger elimination"

function to guarantee all capsules are right ones based on color at rates

up to 3600 per minute

Table 4.7 summarizes how to identify potential applications for machine vision Unquestionably, any operation can identify opportunities for machine vision by performing an introspective examination of its operations Table 4.8 summarizes the benefits that can accrue; these benefits can be the basis for justifying the purchase of machine vision The adoption of this technology, with the result of the objective 100% inspection of products, will cut costs, improve quality, reduce warranty repairs, reduce liability claims, and improve consumer satisfaction - all components in an improved profit picture

Table 4.7 Identifying Applications

1 Lowest value added

2 Process control

3 Separate scrap that can be reworked

4 Avoid expensive hard tooling

5 Avoid equipment breakdown

6 Avoid excess inventory

7 Hazardous environment

8 Operator limitations essential

Page 69

Figure 4.31Vision system from Systronic performs on-line inspection of diapers

to verify presence of all features

Page 70

Figure 4.32Vision system from Avalon Imaging verifying empty state of plastic injection

molding

Page 71Table 4.8 Machine Vision Benefits and Justification Summary

Economic Motivations

To reduce costs of goods manufactured by:

(a) Detecting reject state at point of lowest value added

(b) Automating to reduce work-in-process inventory

(c) Saving on tooling and fixturing costs

(d) Being able to separate scrap than can be reclaimed from that which cannot

(e) Providing early warning to detect incipient reject state to reduce scrap

(f) Reducing scrap and reworking inventory costs

(g) Reducing in warranty repairs, both in the field and returned goods

(h) Reducing service parts distribution costs

(i) Reducing liability costs

(j) Reducing liability insurance

(k) Improving production yield

(l) Reducing direct and indirect labor and burden rate

(m) Increasing equipment utilization

(n) Reducing setup time

(o) Reducing material handling cost and damage

(p) Reducing inventory

(q) Reducing paper

(r) Eliminating schedule upsets

Quality Motivations

To improve quality by:

(a) Conducting 100% inspection versus sample inspection

(b) Improving effectiveness of quality check to improve goods shipped and thereby improving customer

satisfaction

(c) Providing predictability of quality

People Motivations

(a) Satisfy OSHA

(b) Remove from hazardous, unhealthy environment

(c) Avoid contaminants in clean room

(d) Avoid strenuous task

(e) Avoid labor turnover and training costs

(f) Avoid need to hire for seasonal work

(g) Eliminate monotonous and repetitive job

(h) Expedite inspection task that is production bottleneck

(i) Reduce need for skilled people

(j) Avoid errors due to operator judgment, operator fatigue, operator inattentiveness, and operator oversight

(k) Improve skill levels of workers

Substitute capital for labor in expansion mode

Automate record keeping and capture statistics quicker

Feedback signals based on trend analysis to control manufacturing process

Function as "eyes" for automation

Enhance reputation as quality leader

Accelerate response to design changes

Get new technology into business

Page 72

References

Birnbaum, J., "Toward the Domestication of Microelectronics," Computer, November 1985.

Duncan, L S., and Bowen, G L., "Boosting Product Quality for Profit Improvement, "Manufacturing Engineering,

Society of Mechanical Engineers, April 1984

Gevarter, W B., "Machine Vision: A Report on the State of the Art," Computers in Mechanical Engineering, April

1983

Kanade, T., "Visual Sensing and Interpretation: The Image Understanding Point of View," Computers in Mechanical Engineering, April 1983.

Lerner, E J., "Computer Vision Research Looks to the Brain," High Technology, May 1980.

Lowe, D G., "Perceptual Organization and Visual Recognition," National Technical Instrumentation Service

Document AD A-150826

Page 73

5—

Machine Vision:

Introductory Concepts

Machine vision all begin with an image - a picture In many ways the issues associated with a quality image in

machine vision are similar to the issues associated with obtaining a quality image in a photograph In the first place, quality lighting is required in order to obtain a bright enough reflected image of the object Lighting should be

uniformly distributed over the object Non-uniform lighting will affect the distribution of brightness values that will be picked up by the television camera

As is the case in photography, lighting tricks can be used in order to exaggerate certain conditions in the scene being viewed For example, it is possible that shadows can, in effect, include high contrast information that can be used to make a decision about the scene being viewed

The types of lamps that are used to provide illumination may also influence the quality of the image For example, fluorescent lamps have a higher blue spectral output than incandescent lamps While the blue spectral output is more consistent with the spectral sensitivity of the eye, higher infrared output is typically more compatible with the spectral sensitivity of solid state sensors that are used in machine vision

It has been found that the sensitivity of human inspectors can be enhanced as a consequence of using softer lighting or fluorescent lamps with gases that provide more red spectral output; so too it may also be the case in machine vision That is, that the lamp's spectral output may influence the contrast associated with

Page 74the specific feature one is attempting to analyze This has been demonstrated in the case of many organic products

As in photography, machine vision uses a lens to capture a picture of the object and focus it onto a sensor plane The quality of the lens will influence the quality of the image Distortions and aberrations could effect the size of features

in image space Vignetting in a lens can affect the distribution of light across the image plane Magnification of the lens has to be appropriate for the application As much as possible the image of the object should fill the image plane

of the sensor

Allowances have to be made for any registration errors associated with the position of the object and the repeatability

of that positioning The focal length and aperture have to be optimized in order to handle the depth of field associated with the object

The imaging sensor that is used in the machine vision system will basically dictate the limit of discrimination of detail that will be experienced with the system Imaging sensors have a finite number of discrete detectors and this number limits the number of spatial data elements that can be processed or into which the image will be dissected In a typical television-based machine vision system today the number of spatial data points is on the order of 400 to 500 horizontal

× 400 to 500 vertical

Basically, what this means basically is that the smallest piece of information that can be discriminated is going to be a function of the field of view Just like in photography, one can use panoramic optics to take a view of a mountain range, and although a family might be in the picture in the foothills of the mountains, it is unlikely that you would be able to discriminate the family in the picture On the other hand, using a different lens and moving closer to the family, one would be able to capture the facial expressions of each member, but the resulting picture would not include the peaks of the mountains

So, for example, given that an application requires a one-inch field of view, and a sensor with the equivalent of 500 spatial data points is used, one would have a spatial data point that would be approximately 002 inches on the side Significantly, the ability of machine vision today to discriminate details in a scene is generally better than the size of a spatial data point.

In a manner basically analogous to how an eye can see stars in a night sky because of the contrast associated with the star light, so too in machine vision techniques exist which allow systems to be able to discriminate details smaller than

a spatial data element Again, contrast is critical The claims for subpixel sensitivity vary from vendor to vendor and depend very much on their execution and the application

In all machine vision systems up until this point in our discussion, the information or the image has been in an analog format For a computer to operate on the picture the analog image must be digitized This operation basically consists

of sampling at discrete locations along the analog signal that corresponds to a plot of time vs brightness, and

quantizing the brightness at that sample point

Page 75The actual brightness value is dependent on: the lighting, the reflective property of the object, conditions in the

atmosphere between the lighting and the object and between the object and the camera, and the specific detector

sensitivity in the imaging sensor Most vision systems today characterize the brightness into a value of between 0 and

255 The brightness so characterized is generally referred to as a shade of gray

For the most part today, machine vision systems are monochromatic Consequently, the color may also be a factor in the brightness value That is, it is possible to have a shade of red and a shade of green (and so on) all of which would have the same brightness value In many cases where color issues are a concern, filters are used in order to eliminate all colors that are not of interest to the particular application In this way the gray shades are an indicator of the

saturation level associated with a specific color Color cameras can also be used to acquire the data and segmentation based on the specific color enabled

At last we have a picture that has been prepared for a computer In most machine vision systems today, the digitized image is stored in memory that is separated from the computer memory This dedicated memory is referred to as a frame store - where frame is synonymous with the term used in television to describe a single picture In some cases the dedicated hardware that includes the frame store also includes the analog-to-digital converter as well as other electronics to permit one to view images after processing steps have been conducted on the image to view the effects

of these processing procedures

Now the computer can operate on the image The operation of the computer on the image is generally referred to as image processing In addition to operating on the image, the computer is also used to analyze the image and make a decision on the basis of the analyzed image and perform an operation accordingly What is typically referred to as the vision engine part of the machine vision system is the combination of image processing, analysis and decision-making techniques that are embodied in the computer

A good analogy can be made to a toolbox Virtually all machine vision systems today include certain fundamental tools, much like a hammer, screwdriver or pliers Beyond these, different suppliers have developed additional tools, more often than not driven by a specific class of applications Consequently the description frequently given for

machine vision as being an "idiot savant" is quite apropos That is, most of the platforms are brilliant on one set of applications but "idiots" or truly not the optimal for other applications

It is important, therefore, to select the vision platform or toolbox with the most appropriate tools for an application Significantly, no machine vision systems exist today that come anywhere near simulating the comprehensive image understanding capabilities that people have It is noted that for many applications many different tools will actually do the job and in many cases without sacrificing performance On the other hand, in some cases while the tools appear to

do the