Working with a machine-vision-system integrator

A discussion with John Nagle, Nagle Research

VSD: What sort of systems or services does Nagle Research provide?
Nagle: While many companies are involved with 2-D machine vision...

VSD: What sort of systems or services does Nagle Research provide?

Nagle: While many companies are involved with 2-D machine vision, we have decided to devote ourselves almost exclusively to 3-D machine-vision development. We think this has allowed us to build an expertise and body of experience with 3-D technology that is second to none in the industry. Nagle Research is a SICK (Minneapolis, MN, USA; www.sick.com) vision integrator. We are entirely brand loyal to SICK|IVP vision products, most often using the Ranger series of cameras. We started the company in June 2003.

My business partner, Andy Thyssen, is also a software engineer, and he is the chief technology officer of the company. Our first project and what is generally regarded as our “claim to fame” is the Aurora automated high-speed railroad track-inspection system. In our past lives, we spent more than a decade making video games for Nintendo, Playstation, and others. That experience has been immeasurably valuable in keeping the performance of our systems on the leading edge.

VSD: What sort of questions should be asked when considering the services of a machine-vision system integrator?

Nagle: Suffice it to say, it is impossible to engineer a solution without a thorough understanding of the problem. But to truly know the problem, you have to get past the superficial goals and get to the meat of the challenges that any solution will have to face. There can be many gremlins hiding just below the surface of what seems like an “easy” project.

For example, a candy factory needs a vision system that can count jellybeans moving down a conveyor belt. That’s the superficial goal. Obviously this is a very straightforward task for a vision system to accomplish. To be able to intelligently plan a solution, however, requires much more information. What should the system do with the count? Does it need to trigger a signal when a certain count is reached? Does it need to communicate with a PLC? What if a jellybean is malformed, does it count? And how does the system determine what is a “good” jellybean? How fast are the jellybeans moving? Do we need to count the individual colors? What are the space considerations for the vision system? This is very “goal-oriented” fact-finding research, and so this sort of questionand- answer probing can be done even by nontechnical people. Once all of the major and minor goals are known, then it is straightforward to isolate the specific disciplines and skill sets required to make the project a success.

VSD: So what can be done in-house by a company?

Nagle: Evaluating one’s own capabilities or the capabilities of company staff members is the next step in deciding how much, if any, of the project can be done in-house. If the project can be accomplished with off-the-shelf vision solutions or relatively simple smart cameras and only minor external connectivity is required, then the chances of being able to do this are good. If complicated record keeping, PLC connectivity, or advanced image-processing algorithms are required, it is almost certain that a third-party vision-system integrator with software-development capability will be necessary.

Different skills are required to integrate vision systems of varying degrees of complexity. Even a good list of necessary skills cannot be comprehensive and should be treated only as a guideline or rule of thumb.

VSD: What are the implications of working with a 2-D vs. 3-D system?

Nagle: Most people who have experience with vision are likely to have worked only with 2-D systems. Far less common are those who have worked with 3-D. Twodimensional systems deal with color and contrast; three-dimensional systems deal with materials and geometry. The share a lot of the same concepts, but, in general, 3- D is more difficult to implement. This is because now we are not just dealing with a light and a camera, we have to deal with laser light frequency; beam spread angle and thickness; laser power requirements based on material properties and stand-off distance; ranging algorithms; angular orientation of camera/subject/laser to obtain required accuracy; safety issues related to working with the laser; and coping with less than ideal material properties.

Integrating a SICK-IVC-3D or a Ruler product can mitigate some of these issues, in that the camera lens, laser type, and orientation are fixed at the factory (which also limits to some degree their applicability.) Ranging algorithms and material properties must still be dealt with in any case.

VSD: Is a vision software-development kit difficult to learn?

Nagle: In any nonsmart camera system, the integrator must have a thorough knowledge of the vision hardware SDK (software development kit), including the SDK for the frame grabber if applicable. Speaking in general terms, these are highly nontrivial software toolkits and a deep-rooted foundation in C++ and software development is essential. Even with the requisite C++ experience, the SDK itself—like any complex system—has a learning curve. If the project can absorb the extra time and cost associated with becoming proficient with the SDK, then it is very feasible.

VSD: What are the benefits of third-party integration?

Nagle: Any competent vision integrator should be able to integrate vision in simple to moderately complex projects. Many vision integrators do not have great depth in software and electrical engineering, and so for many the more complicated vision projects are beyond them. When choosing vision integrators do not have great depth in software and electrical engineering, and so for many the more complicated vision projects are beyond them. When choosing an integrator, it becomes very important to match the skills they bring to the table with the skills that will be required. Dealing with an integrator can save an enormous amount of time and development effort. In many cases, experienced integrators have saved companies from spending hundreds of thousands of dollars on inappropriate equipment and software.

For example, we were asked by a railroad-equipment manufacturer to provide consultation as to what camera would be required for a 2-D high-speed-railroad-inspection system. The company had already spent many thousands of dollars on image-processing software to locate defects in crossties using 2-D imaging techniques. The problem was that their approach had not accounted for surface stains, sealant, and debris confusing the analysis software. We ultimately concluded that a 3-D solution was more appropriate for this application and developed a Ranger-based solution that handles these material properties nicely.

VSD: When working with a system integrator, what are you paying for?

Nagle: Speaking only for Nagle Research, in most cases vision projects are quoted on a flat fee basis. Usually the process is phone conference to discuss the challenges and goals; if possible, samples are sent for testing and proof-of-concept; and if the project proves solvable, we submit a proposal.

With projects whose goals are a moving target--for example, additional defects to detect or additional accuracy requiring more cameras--there will most likely be proposed a flat fee for a defined scope of work and a standard hourly fee for work that is out of scope. The proposal will include time for travel, but the travel expenses are billed separately.

For our fee, the client receives our professional consultation, software and electrical engineering resources, and, in the end, a solution that meets their requirements. In most cases, unless specifically agreed to, the client does not get source code to the final solution. In some arrangements we will relinquish source code for the application, for example, their user interface and project-specific algorithms. Our proprietary Javelin Vision Engine, however, remains closed source. Javelin is the 3-D technology infrastructure to help us in developing more robust vision systems

VSD: What are the fundamental questions to ask before calling an integrator?

Nagle: The basic questions that need to be answered before an integrator is called are • Is the project outside the scope of in-house capabilities? • Is the company open to using third-party integrators? • What is the price of failure or delays arising from lack of internal experience? • Is there a budget for vision that includes third-party integration? • Is there likelihood that given a workable solution within budget, the project would proceed?

If the answer to all of these is “yes,” then most any integrator would be willing to take the challenge. A competent vision integrator is the key to successfully deploying a machine-vision system. Whether or not that expertise comes from within or from a third party is a decision the client ultimately will have to make. The most important thing to keep in mind is that in any event, a broad skill set and expertise in a variety of disparate disciplines will be required to complete the project on time and on budget.

Inspection systems focus on performance, robustness, and stability

A discussion with Robert Massen, Baumer Inspection

VSD: What systems or services does your company provide? What is the origin of your company?
VSD: What systems or services does your company provide? What is the origin of your company?

Massen: Our company started via a management buyout from a former for-profit institute, which I founded together with the Steinbeis Foundation at the University of Applied Sciences in Konstanz, Germany, around 1982. We have a history of 25 years in machine vision, starting with a large number of industrial customer projects and focusing now on in-line inspection of aesthetic surfaces: multicolored, patterned, and textured products such as laminate floorings, decorated furniture panels, ceramic tile inspection. We design, install, and service worldwide in-line inspection, sorting, and process-monitoring vision systems and are a leader in the field of automated laminate flooring inspection.

VSD: What technologies and components do you use in machine-vision-related applications? How often do you evaluate competing technologies?

Massen: To simulate the very peculiar human perception of decorated multicolored surfaces and at the same time detect physical defects such as scratches, bubbles, bad transparent protection layers, chipped edges, and so forth, we combine multiple camera/ illumination systems into our multisensorial inspection technology, including color linescan cameras with diffuse ultrastable illumination, black-and-white linescan cameras and directed light (4k and 6k, mostly), and spectrally tuned linescan cameras and specific wavelength illumination for the inspection of transparent protective layers.

We prefer Camera Link frame grabbers, possibly with a bit of integrated FPGA preprocessing. We use a PC cluster architecture for achieving the high computing power required for that type of color and texture processing. To be flexible to the ever-changing aesthetic designs of very creative artists, we use almost 100%software-basedimage processing with our own libraries of image-processing algorithms.

Almost half of our staff of 50 employees are software and vision specialists, who do checkout possibly competing technologies. We have a continuing education philosophy, sending our vision specialists to conferences and having some of them working as part-time Ph.D. students.

VSD: How do you evaluate the performance of the few color linescan cameras on the market? Which cameras do you use in your designs?

Massen: A reasonably priced, ruggedized color linsescan camera with high geometric resolution, fast linescan frequency, very low color seams, operating at variable product speeds, and radiometrically stable under severe temperature fluctuations of an industrial production line is still a bit of a dream. We never trust the published technical specifications of the camera manufacturers and even less those communicated by distributors, but we do tests these cameras extensively in our labs. We often discover subtle, but technically important flaws or items missing in the published specs. We always have to do a careful selection of appropriate lenses that are hard to find, both for trilinear and especially for 3CCD prismatic color linescan cameras.

VSD: How do you design your systems for OEM product obsolescence?

Massen: As our systems are highly modular PC-and software-based architectures, we have no problems in adapting to a new generation of motherboards, multicore processors, or new operating systems. We program in a very modular way in standard C/C++, separating software and hardware; so changing to a new camera or frame grabber does not pose any problem. Our customers appreciate this guarantee of long operating life and of a familiar PC and Windows environment, even if arranged as clusters of up to 16 networked PCs.

VSD: In which areas of the industry do you see the most growth? What are users demanding from you in the design of new systems?

Massen: The broad field of nonindustrial vision markets such as security, traffic, and toll control; electronic driver assistance; postal distribution; and logistics will grow at a faster rate than the vision market for machine-vision systems operating in the production line. These systems will use a lot of components, software, and knowledge existing in the machine vision scene. Some established machine-vision companies such as Vitronic (Wiesbaden, Germany; www.vitronic.com) are expanding quickly into these new markets parallel to their ongoing machine-vision activity.

A ColourBrain Laminate Inspection System from Baumer Inspection examines laminated fullboards at a Pergo production facility in Garner, NC, USA. Our customers are using faster and faster running, highly automated, almost unmanned production plants for an evergrowing variety of decorations and for production batches ranging from hundred of thousands to batch size one (such as in automatic kitchen-producing plants).

They ask us to offer them simple-to-use and stable inspection systems for their highly complex inspection tasks. For automatic very-high-speed visual sorting, they need very low overdetection rates (wrongly classifying good products as being bad), fast training for new décors, and an integrated automatic process-monitoring and quality management system.

VSD: How will OEM components targeted toward machine-vision applications have to change to meet future needs?

Massen: In our specific segment of inspection in production lines, the focus is more on technical performance, robustness, and stability then on price alone. We would appreciate better color linescan cameras operating without false-color seams at wide observation angles to decrease the height of our systems. We would also appreciate the development of cameras that integrate several narrow-spectral-band linescan sensors and fast 3-D sensors in one camera body.

VSD: Could you compare the machine-vision markets in different industry segments in Europe and the rest of the world?

Massen: The European machine market is highly dominated by German companies that produce some 82% of the European turnover. The German machine-vision scene is extremely active, both in the field of application-specific vision systems and also for vision components (cameras, frame grabbers, illumination systems, and software libraries). The specific excellence of the German “Sondermaschinenbau” (specialized highly automated production machinery)is closely related to German machine-vision companies, which has given a definite push to both. A good example of this process is the recent acquisition by the German Weinig Group, a leader in wood-processing machinery, of LuxScan Technologies, a wood scanner company in Luxembourg.

I do see a good and hard-to-copyf uture for similar marriages between advanced production machinery companies and vision companies. At the same time it is astonishing to see German companies such as Basler (Ahrensburg, Germany; www. basler-vc.com) exporting high volumes of cameras produced in high-wage Germany to low-wage China, again a proof that technical excellence and professionalism can compete with low salaries.

VSD: Could you discuss the impact of working with Baumer? What has that meant for your business? How are you now organized?

Massen: The Swiss Baumer Group, a family owned group of some 35 companies with a total of 2000 people, invited MASSEN machine-vision systems as a shareholder in 1992. Baumer was a great help in moving our activity from a more project-based institute activity to a product-based company by focusing on a small number of markets. The 100% integration into the group is therefore not a surprising move but a natural development. The rebranding into Baumer Inspection increases the visibility of Baumer as a unique group of companies producing the total spectrum of noncontact sensors, from classical proximity to vision sensors, intelligent cameras, and application-specific machine-vision systems.

We are a member of the vision technologies business unit of Baumer, which employs some 250 people. This is very broad range of expertise that helps us both in view of vision components and technologies available from the group and as a worldwide machine-vision business supported by the presence of the group´s subsidaries.

ROBERT MASSEN turned a for-profit research institute in image processing into the private MASSEN machine vision systems GmbH, which recently became part of the Baumer Group and was rebranded as Baumer Inspection. Editor in chief Conard Holton spoke with him about integrated system design.