Complex Thinking

A discussion with Katrin Pape, CTMV

Katrin Pape is cofounder and managing director of CTMV (Consulting Team Machine Vision; Pforzheim, Germany; www.ctmv.de). She has worked in numerous positions in the machine-vision industry.

VSD: Please describe your company and its services.
Pape: CTMV is a solution provider with experience in the field of image processing. Its founders draw on expertise in developing new application areas and complex application scenarios and their implementation in workable business solutions. Applications are focused on surface inspection of visually difficult materials such as glass, ceramics, various metals, plastic tubes, and foils.

In addition, we provide precise dimensional- accuracy verification, mainly for the stamping industry, and on presence/absence checks in applications such as the assembly of gear shafts, as well as position detection of moving and/or complex parts for robots and handling. CTMV offers business solutions for quality assurance in the fields of medicine, electronics, and automotive, as well as for process optimization of manufacturing in the fields of metal working, extrusion, and foil/paper production.

VSD: What technologies and components do you use for your applications? How often do you evaluate competing technologies?
Pape: Depending on the inspection task and its general framework, we use area- or linescan camera technology combined with appropriate standard interfaces based on camera buses or network technology, as well as PC or embedded compact vision systems. We continuously evaluate new products and work in close cooperation with standard component suppliers. The decision on whether new products will finally be implemented in standard applications is based on their specific benefit and on whether they help solve problems in a more reliable, robust, and possibly cost-efficient way.

Application-specific software with straightforward user interfaces and reliable intelligent algorithms for feature extraction and analysis will usually be implemented by CTMV. Developing field-specific analysis software and tools with a minimum set of parameters but a maximum of intelligence and performance is one of our core competencies.

VSD: How do you approach a new application? Do you work with OEMs or other system integrators?
Pape: Conceptual design of new applications is the key intellectual property that distinguishes us from our competitors. With our broad experience basis as a team, we continue to be able to design new approaches and concepts. This starts by determining the appropriate method of image acquisition, as well as highlighting test criteria with the adequate optical and illumination setup. The process includes developing reliable and robust methods for analysis of the image content, and continues to integration into the process chain with adequate signal and data exchange.

Key requirements are usually specified by customers or mechanical engineers. The diversity of customer and field-specific needs—in each case based on a combination of image processing and process engineering— opens up completely new applications. One example is a system for continuous tube inspection during extrusion, with detection and classification of process-related defects combined with special failure management and alerting. Another is quality control of metallic gear wheels during production, detecting scratches, cracks, and broken parts (see figure).

VSD: How do you design your systems for OEM product obsolescence?
Pape: Our safeguard here is working exclusively with industry standards. Components are exchangeable quickly and easily, so our customers get continuous, reliable functionality. We are independent of operating systems because we cooperate with reputable partners such as National Instruments (Austin, TX, USA; www.ni.com). These companies develop and provide system and field-independent standard components that meet industrial camera interface and automation standards.

VSD: What are users demanding from you in the design of new systems?
Pape: The general expectation is the application of modern, open standard technologies. Based on the latest developments on the components market and the way we integrate these trends into our system concepts, we meet these expectations in every respect.

The bigger challenge, however, is the balancing act that is needed to cover the span between the requirements for maximum identification performance—frequently using highly complex methods of analysis— and the simultaneous required ease of use. Examples of this are the stamping industry, as well as packaging and food, or presence/absence checks and version monitoring.

In these cases, the variety of parts to be inspected is high, and operators want to be able to create test plans for modified or new products themselves. This requires robust test cells on the one hand, and on the other hand adaptable software with respective part management and the right balance between parameterization possibilities and hidden advanced functionality.

To meet these requirements, we generally develop customer-specific user interfaces and implement easy-to-use setup and inspection plan editors with a minimum set of algorithms, tailor-made for the respective application or field of industry. We make a point of thinking in and implementing the language of the respective user or application environment and not that of image processing. Based on these key strategies, we provide users with the necessary flexibility without requiring them to have specific knowledge of image processing.

VSD: How will OEM components targeted toward machine-vision applications have to change to meet future needs?
Pape: Scalability and adaptability are required not only for software, but also for system bases. We often encounter the issue that initial specifications are later enhanced, further inspection tasks are added, or solutions have to be adapted to specific customer environments—all while keeping development efforts as low as possible. The ongoing standardization in the field of camera interfaces is an essential step and key factor in this respect. It enables us to meet these requirements today and should definitely be pursued further into the future.

Another key point is that, as a rule, vision systems need to communicate with a complex network of instrumentation, control, and drive systems. The direct combination of industrial interfaces for process communication such as Ethernet, digital I/O, and so forth, with image-processing components is beneficial for us. Application of various modules, including those from different suppliers, can be minimized or might not be necessary at all. This saves time and limits technical risks.

Examples are modules that combine the camera bus with FPGA-based, digital I/O. The camera bus ensures the image acquisition, while the complete timing, trigger handling, and partly complex encoder handling in real time are done by the I/O module. In this way, and with the systems capable of being operated within networks, we can now solve 95% of inspection tasks using area-scan cameras. We would wish for additional, similar technologies—for example, with linescan or network-based camera interfaces (GigE).

VSD: How do you think that the machine vision market differs in Germany from that in other parts of Europe or North America? What changes have you seen recently in the German market?
Pape: Our customers usually are machinery manufacturers who provide sales and servicing of complete machinery and equipment including optical inspection systems. Machinery incorporating CTMV vision systems has been installed all over North America and Europe. General trends are global. However, in Germany, there is intense competition between suppliers of components and system integrators. On the other hand, mechanical engineers increasingly ask for industrial image processing solutions, identifying these as substantial competitive edge. In general, we see a market differentiation toward plug-and-play solutions that are easy to configure and will not require system integrator services in the future. At the same time, some customers are interested in solutions for new, complex inspection tasks that have not been tackled so far. This is where we as CTMV can provide know how.

VSD: What new markets and technologies do you see for CTMV in the future? Pape: In the next few years, we see continuous opportunities for growth. CTMV will focus on further development of business solutions in medical, electronics, plastics, and automotive markets. And, we will develop new, innovative image-processing solutions within the scope of strategic partnerships with mechanical-engineering companies.

Developing new machine-vision software applications

Christian Demant, NeuroCheck and Industrial Vision Systems, on the demands of advanced development platforms


The fast development cycles in the IT industry result in permanent pressure on machine-vision software developers to update their knowledge. Integral parts of that continuous learning process are new networked system architectures and software-development tools.

Advanced new development platforms like .NET provide the option to easily combine software modules written in several different programming languages. This supports the development in teams having different professional backgrounds. Therefore, developers managing these teams must have knowledge in all these programming languages.

The availability of the latest multicore CPU technology demands multithreaded software to take advantage of this new processing power. Enabling software for parallel computation adds a totally new level of complexity to the development process. The normal programming approach, being used for dozens of years, is now, in many cases, obsolete. The synchronization of parallel executing threads leads to new error situations, which are extremely difficult to predict and to simulate in advance.

In the future I see growing importance of an in-depth understanding of software design patterns. The growing size and complexity of machine-vision software applications requires a much more systematic approach during the design and planning stage. Even 10 years ago a software developer started the implementation a couple of hours after discussing the specification with his sales department.

The software-development process has moved toward something comparable to a team of architects in charge of a complex facility. The design of the building and the drafting of the plans and specifications are the main intellectual tasks and take a big part of the development time. The implementation afterward is handcraft, but both jobs together require skilled teams. Lone fighters have no chance of survival.

Christian Demant is general manager, NeuroCheck, Stuttgart, Germany, and
Director, Industrial Vision Systems, Kingston Bagpuize, UK

Then and now: 20 years of machine-vision system integration

David Dechow, aptúra Machine Vision Solutions, on the changing role of the machine-vision system integrator

Within the context of a machine-vision application, integration quite simply is where someone has to actually make things work. With that definition, the role of the machine-vision integrator has changed little from “ancient times,” when we interfaced vidicon cameras with plodding computers to check the presence of an object based on 32 levels of gray. With scant few exceptions, today’s machine-vision devices do not yet arrive from the manufacturer shrink-wrapped, fully configured, ready to plug in, turn on, and perform an inspection. Machine-vision technology is a combination of diverse components that must be correctly selected based upon the needs of the application, competently installed, programmed, or configured to provide a robust result, then tested to ensure reliability that will withstand a production environment. Bottom line: a machine-vision application still must be “integrated”; someone has to “make it work.”

What is new is that machine vision has become a technology that today is significantly more accessible to the plant engineer than it was even several years ago. Yet the need and demand for the machine-vision system integrator is as strong as ever. What has changed ever so slightly is the perceived ROI or value of the integration partner. At one time the machine-vision system integrator was absolutely required even to consider an inspection project. The prevailing perception now is that the inspection task could be successfully done “in-house”; but it is more efficient and effective to use an outside integration resource for machine vision. A parallel situation occurred in the maturation of the PLC integration market, where today it is very common for a company to hire contractors to develop and maintain machine logic rather than have a team of company experts.

Looking ahead, the machine-vision integrator likely will continue to be an engineering resource for end users, machine builders, and other integrators, providing services on a contract or time-and-materials basis. Integrators will be called upon to execute more complicated inspection systems and will need to maintain relatively higher levels of machine-vision expertise. The barriers to entry into the vision integrator marketplace remain low, but the name of the game is efficiency and profitability, and the machine vision integration entity will increasingly need to find economies of scale that will sustain the business model.

Working with a machine-vision-system integrator

A discussion with John Nagle, Nagle Research

VSD: What sort of systems or services does Nagle Research provide?
Nagle: While many companies are involved with 2-D machine vision...

VSD: What sort of systems or services does Nagle Research provide?

Nagle: While many companies are involved with 2-D machine vision, we have decided to devote ourselves almost exclusively to 3-D machine-vision development. We think this has allowed us to build an expertise and body of experience with 3-D technology that is second to none in the industry. Nagle Research is a SICK (Minneapolis, MN, USA; www.sick.com) vision integrator. We are entirely brand loyal to SICK|IVP vision products, most often using the Ranger series of cameras. We started the company in June 2003.

My business partner, Andy Thyssen, is also a software engineer, and he is the chief technology officer of the company. Our first project and what is generally regarded as our “claim to fame” is the Aurora automated high-speed railroad track-inspection system. In our past lives, we spent more than a decade making video games for Nintendo, Playstation, and others. That experience has been immeasurably valuable in keeping the performance of our systems on the leading edge.

VSD: What sort of questions should be asked when considering the services of a machine-vision system integrator?

Nagle: Suffice it to say, it is impossible to engineer a solution without a thorough understanding of the problem. But to truly know the problem, you have to get past the superficial goals and get to the meat of the challenges that any solution will have to face. There can be many gremlins hiding just below the surface of what seems like an “easy” project.

For example, a candy factory needs a vision system that can count jellybeans moving down a conveyor belt. That’s the superficial goal. Obviously this is a very straightforward task for a vision system to accomplish. To be able to intelligently plan a solution, however, requires much more information. What should the system do with the count? Does it need to trigger a signal when a certain count is reached? Does it need to communicate with a PLC? What if a jellybean is malformed, does it count? And how does the system determine what is a “good” jellybean? How fast are the jellybeans moving? Do we need to count the individual colors? What are the space considerations for the vision system? This is very “goal-oriented” fact-finding research, and so this sort of questionand- answer probing can be done even by nontechnical people. Once all of the major and minor goals are known, then it is straightforward to isolate the specific disciplines and skill sets required to make the project a success.

VSD: So what can be done in-house by a company?

Nagle: Evaluating one’s own capabilities or the capabilities of company staff members is the next step in deciding how much, if any, of the project can be done in-house. If the project can be accomplished with off-the-shelf vision solutions or relatively simple smart cameras and only minor external connectivity is required, then the chances of being able to do this are good. If complicated record keeping, PLC connectivity, or advanced image-processing algorithms are required, it is almost certain that a third-party vision-system integrator with software-development capability will be necessary.

Different skills are required to integrate vision systems of varying degrees of complexity. Even a good list of necessary skills cannot be comprehensive and should be treated only as a guideline or rule of thumb.

VSD: What are the implications of working with a 2-D vs. 3-D system?

Nagle: Most people who have experience with vision are likely to have worked only with 2-D systems. Far less common are those who have worked with 3-D. Twodimensional systems deal with color and contrast; three-dimensional systems deal with materials and geometry. The share a lot of the same concepts, but, in general, 3- D is more difficult to implement. This is because now we are not just dealing with a light and a camera, we have to deal with laser light frequency; beam spread angle and thickness; laser power requirements based on material properties and stand-off distance; ranging algorithms; angular orientation of camera/subject/laser to obtain required accuracy; safety issues related to working with the laser; and coping with less than ideal material properties.

Integrating a SICK-IVC-3D or a Ruler product can mitigate some of these issues, in that the camera lens, laser type, and orientation are fixed at the factory (which also limits to some degree their applicability.) Ranging algorithms and material properties must still be dealt with in any case.

VSD: Is a vision software-development kit difficult to learn?

Nagle: In any nonsmart camera system, the integrator must have a thorough knowledge of the vision hardware SDK (software development kit), including the SDK for the frame grabber if applicable. Speaking in general terms, these are highly nontrivial software toolkits and a deep-rooted foundation in C++ and software development is essential. Even with the requisite C++ experience, the SDK itself—like any complex system—has a learning curve. If the project can absorb the extra time and cost associated with becoming proficient with the SDK, then it is very feasible.

VSD: What are the benefits of third-party integration?

Nagle: Any competent vision integrator should be able to integrate vision in simple to moderately complex projects. Many vision integrators do not have great depth in software and electrical engineering, and so for many the more complicated vision projects are beyond them. When choosing vision integrators do not have great depth in software and electrical engineering, and so for many the more complicated vision projects are beyond them. When choosing an integrator, it becomes very important to match the skills they bring to the table with the skills that will be required. Dealing with an integrator can save an enormous amount of time and development effort. In many cases, experienced integrators have saved companies from spending hundreds of thousands of dollars on inappropriate equipment and software.

For example, we were asked by a railroad-equipment manufacturer to provide consultation as to what camera would be required for a 2-D high-speed-railroad-inspection system. The company had already spent many thousands of dollars on image-processing software to locate defects in crossties using 2-D imaging techniques. The problem was that their approach had not accounted for surface stains, sealant, and debris confusing the analysis software. We ultimately concluded that a 3-D solution was more appropriate for this application and developed a Ranger-based solution that handles these material properties nicely.

VSD: When working with a system integrator, what are you paying for?

Nagle: Speaking only for Nagle Research, in most cases vision projects are quoted on a flat fee basis. Usually the process is phone conference to discuss the challenges and goals; if possible, samples are sent for testing and proof-of-concept; and if the project proves solvable, we submit a proposal.

With projects whose goals are a moving target--for example, additional defects to detect or additional accuracy requiring more cameras--there will most likely be proposed a flat fee for a defined scope of work and a standard hourly fee for work that is out of scope. The proposal will include time for travel, but the travel expenses are billed separately.

For our fee, the client receives our professional consultation, software and electrical engineering resources, and, in the end, a solution that meets their requirements. In most cases, unless specifically agreed to, the client does not get source code to the final solution. In some arrangements we will relinquish source code for the application, for example, their user interface and project-specific algorithms. Our proprietary Javelin Vision Engine, however, remains closed source. Javelin is the 3-D technology infrastructure to help us in developing more robust vision systems

VSD: What are the fundamental questions to ask before calling an integrator?

Nagle: The basic questions that need to be answered before an integrator is called are • Is the project outside the scope of in-house capabilities? • Is the company open to using third-party integrators? • What is the price of failure or delays arising from lack of internal experience? • Is there a budget for vision that includes third-party integration? • Is there likelihood that given a workable solution within budget, the project would proceed?

If the answer to all of these is “yes,” then most any integrator would be willing to take the challenge. A competent vision integrator is the key to successfully deploying a machine-vision system. Whether or not that expertise comes from within or from a third party is a decision the client ultimately will have to make. The most important thing to keep in mind is that in any event, a broad skill set and expertise in a variety of disparate disciplines will be required to complete the project on time and on budget.

Inspection systems focus on performance, robustness, and stability

A discussion with Robert Massen, Baumer Inspection

VSD: What systems or services does your company provide? What is the origin of your company?
VSD: What systems or services does your company provide? What is the origin of your company?

Massen: Our company started via a management buyout from a former for-profit institute, which I founded together with the Steinbeis Foundation at the University of Applied Sciences in Konstanz, Germany, around 1982. We have a history of 25 years in machine vision, starting with a large number of industrial customer projects and focusing now on in-line inspection of aesthetic surfaces: multicolored, patterned, and textured products such as laminate floorings, decorated furniture panels, ceramic tile inspection. We design, install, and service worldwide in-line inspection, sorting, and process-monitoring vision systems and are a leader in the field of automated laminate flooring inspection.

VSD: What technologies and components do you use in machine-vision-related applications? How often do you evaluate competing technologies?

Massen: To simulate the very peculiar human perception of decorated multicolored surfaces and at the same time detect physical defects such as scratches, bubbles, bad transparent protection layers, chipped edges, and so forth, we combine multiple camera/ illumination systems into our multisensorial inspection technology, including color linescan cameras with diffuse ultrastable illumination, black-and-white linescan cameras and directed light (4k and 6k, mostly), and spectrally tuned linescan cameras and specific wavelength illumination for the inspection of transparent protective layers.

We prefer Camera Link frame grabbers, possibly with a bit of integrated FPGA preprocessing. We use a PC cluster architecture for achieving the high computing power required for that type of color and texture processing. To be flexible to the ever-changing aesthetic designs of very creative artists, we use almost 100%software-basedimage processing with our own libraries of image-processing algorithms.

Almost half of our staff of 50 employees are software and vision specialists, who do checkout possibly competing technologies. We have a continuing education philosophy, sending our vision specialists to conferences and having some of them working as part-time Ph.D. students.

VSD: How do you evaluate the performance of the few color linescan cameras on the market? Which cameras do you use in your designs?

Massen: A reasonably priced, ruggedized color linsescan camera with high geometric resolution, fast linescan frequency, very low color seams, operating at variable product speeds, and radiometrically stable under severe temperature fluctuations of an industrial production line is still a bit of a dream. We never trust the published technical specifications of the camera manufacturers and even less those communicated by distributors, but we do tests these cameras extensively in our labs. We often discover subtle, but technically important flaws or items missing in the published specs. We always have to do a careful selection of appropriate lenses that are hard to find, both for trilinear and especially for 3CCD prismatic color linescan cameras.

VSD: How do you design your systems for OEM product obsolescence?

Massen: As our systems are highly modular PC-and software-based architectures, we have no problems in adapting to a new generation of motherboards, multicore processors, or new operating systems. We program in a very modular way in standard C/C++, separating software and hardware; so changing to a new camera or frame grabber does not pose any problem. Our customers appreciate this guarantee of long operating life and of a familiar PC and Windows environment, even if arranged as clusters of up to 16 networked PCs.

VSD: In which areas of the industry do you see the most growth? What are users demanding from you in the design of new systems?

Massen: The broad field of nonindustrial vision markets such as security, traffic, and toll control; electronic driver assistance; postal distribution; and logistics will grow at a faster rate than the vision market for machine-vision systems operating in the production line. These systems will use a lot of components, software, and knowledge existing in the machine vision scene. Some established machine-vision companies such as Vitronic (Wiesbaden, Germany; www.vitronic.com) are expanding quickly into these new markets parallel to their ongoing machine-vision activity.

A ColourBrain Laminate Inspection System from Baumer Inspection examines laminated fullboards at a Pergo production facility in Garner, NC, USA. Our customers are using faster and faster running, highly automated, almost unmanned production plants for an evergrowing variety of decorations and for production batches ranging from hundred of thousands to batch size one (such as in automatic kitchen-producing plants).

They ask us to offer them simple-to-use and stable inspection systems for their highly complex inspection tasks. For automatic very-high-speed visual sorting, they need very low overdetection rates (wrongly classifying good products as being bad), fast training for new décors, and an integrated automatic process-monitoring and quality management system.

VSD: How will OEM components targeted toward machine-vision applications have to change to meet future needs?

Massen: In our specific segment of inspection in production lines, the focus is more on technical performance, robustness, and stability then on price alone. We would appreciate better color linescan cameras operating without false-color seams at wide observation angles to decrease the height of our systems. We would also appreciate the development of cameras that integrate several narrow-spectral-band linescan sensors and fast 3-D sensors in one camera body.

VSD: Could you compare the machine-vision markets in different industry segments in Europe and the rest of the world?

Massen: The European machine market is highly dominated by German companies that produce some 82% of the European turnover. The German machine-vision scene is extremely active, both in the field of application-specific vision systems and also for vision components (cameras, frame grabbers, illumination systems, and software libraries). The specific excellence of the German “Sondermaschinenbau” (specialized highly automated production machinery)is closely related to German machine-vision companies, which has given a definite push to both. A good example of this process is the recent acquisition by the German Weinig Group, a leader in wood-processing machinery, of LuxScan Technologies, a wood scanner company in Luxembourg.

I do see a good and hard-to-copyf uture for similar marriages between advanced production machinery companies and vision companies. At the same time it is astonishing to see German companies such as Basler (Ahrensburg, Germany; www. basler-vc.com) exporting high volumes of cameras produced in high-wage Germany to low-wage China, again a proof that technical excellence and professionalism can compete with low salaries.

VSD: Could you discuss the impact of working with Baumer? What has that meant for your business? How are you now organized?

Massen: The Swiss Baumer Group, a family owned group of some 35 companies with a total of 2000 people, invited MASSEN machine-vision systems as a shareholder in 1992. Baumer was a great help in moving our activity from a more project-based institute activity to a product-based company by focusing on a small number of markets. The 100% integration into the group is therefore not a surprising move but a natural development. The rebranding into Baumer Inspection increases the visibility of Baumer as a unique group of companies producing the total spectrum of noncontact sensors, from classical proximity to vision sensors, intelligent cameras, and application-specific machine-vision systems.

We are a member of the vision technologies business unit of Baumer, which employs some 250 people. This is very broad range of expertise that helps us both in view of vision components and technologies available from the group and as a worldwide machine-vision business supported by the presence of the group´s subsidaries.

ROBERT MASSEN turned a for-profit research institute in image processing into the private MASSEN machine vision systems GmbH, which recently became part of the Baumer Group and was rebranded as Baumer Inspection. Editor in chief Conard Holton spoke with him about integrated system design.