This guide gives an step by step introduction in industrial hyperspectral imaging

This document guides the reader through a complete hyperspectral imaging workflow beginning with the setup of components and ending up with the inspection of a color stream to discriminate taught plastics.

To get this tour practical, the differentiation of three plastic plates by means of its molecular fingerprint is shown. In this example hyperspectral line-scan technology (VNIR range) is used to differentiate plastic plates of white color.

Step 1 - Setup a hyperspectral system

Before start, check to have all components available:

• CCI-compliant hyperspectral camera
• Illumination capable for hyperspectral measurements
• Calibration target
• Perception Studio program
• When real-time processing is needed, a CC-compliant streaming adapter (e.g. an industrial PC with the Perception Core program running on it)
• A black marker (needed for setting up the sharpness)
• A good absorbing background material often is helpful (e.g. a black foam)
• Reference material for first measurements (e.g. plastic parts)

Setup hardware and software

In this section the setup of hardware and software components for industrial hyperspectral imaging is described.

Deciding on the system purpose

With Perception Studio and Perception Core, two general use cases are possible:

1. Perception Studio allows to record data from supported hyperspectral cameras, store and manipulate this data and finally create models, which can be applied to this data. These models wil be able to convert hyperspectral data into image data, which makes application relevand information visible. For instance such a model might show an image where good materials are colored green and contaminants are colored red. 
2. Perception Core can run those models, which were generated by Perception Studio in high performance, using GPU acceleration. Thus, a deployed system may consist of a powerfull hardware (with GPU) where a Perception Core software is installed and one or more models that define the system's behavior. All running Perception Core instances in a computer network can be controlled and set up by a Perception Studio application in the same network. 

Setup the hardware components

Mount the camera and lighting so the cameras field of view gets illuminated. It is good practise to use a black background material (e.g. a black foam).

Hint: Be sure the illumination brings light not only from one side onto the line of inspection. Shadows will harm the measurement quality. Best is to use a diffuse illumination or to illuminate from 2 sides.
Be aware about the influence of alternating current to the measurement results. The alternation can cause a ripple in the measured intensity over time. E.g. use an illumination powered by direct current to avoid this disturbing influence in the data.

Keep the minimum measurement distance of your camera in mind - e.g. ask your camera provider about this measure.
Hint: A distance smaller than this will result in not sharp camera images.

Streaming of molecular information is aimed:
Connect the camera with a CCI-compliant streaming adapter (Perception Core on it). Connect your PC to the streaming adapter.

Streaming is not needed:
Connect the camera to your PC.

 Measurement setup

Measurement setup
Specim FX10 hyperspectral camera, 2-sided Halogen illumination and a white tile in the camera's field of view

Setup the camera

Depending on the type of camera, several steps might be needed before a connection to the camera is possible and all functions of the camera can be used.

For GenICam compliant cameras such as Specim FX10e, FX17e and AlliedVision G008, see this manual:  How to configure automatic detection for GenICam cameras

For CameraLink cameras such as Specim FX10, FX17 and AlliedVision CL032, install the grabber drivers and software and verify that you have full access to the camera before proceeding

Setup other devices

If you also have other devices like Specim's linear table, you also need to install the LUMO software by Specim (Installation of LUMO). However, LUMO is not needed if you only want to work with the cameras.

Setup the software components

Make sure the camera is connected properly (see previous section).

→ Install the Perception Studio program on your PC. 

→ If you want to also setup a Perception Core program, you can install it on the same PC as Perception Studio or on another PC in the same network  (e.g. on the industrial PC). However, the PC needs a GPU, which is supported.

→ Start the Perception Studio program and make sure the hardware was detected properly.

Hint: In case you are connected to the streaming adapter, the Perception Core icon will get shown in the device section of the user interface. In case of a direct connection to the camera, the cameras vendor icon will get shown in the user interface.

Setup the acquisition

Setup the optics and acquisition parameters.

Setup the camera's acquisition parameters

→ Open Perception Studio's Setup perspective and start the live visualization of camera data.

→ Select the data dimension "spatial" and inspect the camera live data in the view.

→ Control the camera's acquisition parameter to find a proper saturation of the sensor.

Inspect the sensor's saturation for the "white" situation (light gets reflected from the calibration target) vs. the "dark" situation (e.g. the cap is on the lens and blocks the optical path).
Be sure the signal dynamic is reasonable high. Avoid under or over-saturated sensor regions.

Setup the camera's image sharpness

→ Open Perception Studio's Setup perspective and start the live visualization of camera data.

→ Select the data dimension "spatial" and inspect the camera live data in the view.

→ Put a black marker into the camera's field of view (e.g. onto the calibration target) and inspect obtained live data.

→ Adjust the optical setup (e.g. the lens) to achieve the maximum spatial sharpness.

Try to get an imaging of the marker with best contrast possible - so the marker appears with highest sharpness in the cameras image.

 Sharpness ok vs. not ok

Sharpness not ok	Sharpness ok
A black marker was put onto the calibration target (line of inspection).
The fore lens need to get adjusted.	The fore lens was set properly.

Setup the measurement

The interaction of light with objects in general is diverse. One part is reflected from the object, another one get absorbed by the object and in case of a thin object, one part get transmitted though the object.
In industry, very often, molecular properties of objects are attempted to get measured by the study of reflected light per wavelength relatively to a known target.

Setup the system to measure relatively to a target.

Setup the balancing

→ Open Perception Studio's Setup perspective and start the live visualization of camera data.

→ Put the calibration target into the camera field of view and inspect obtained camera data.

→ Perform dark balancing

→ Perform white balancing

When inspecting the "white" image, be aware the cameras sensor is saturated properly. Under- or over-saturated pixel regions will cause measurement errors.

Block the optical path of the camera (e.g. put the cap onto the fore lens) and click on Record Dark Image button.

Be sure the camera sensor is properly saturated than click on the Record White Image button.

When done, check the "balanced" option on the top of the graph and inspect balanced live data of the camera.

Step 2 - Acquire hyperspectral data

This chapter guides you through the acquisition process of hyperspectral data.

Be sure your hyperspectral system was set up properly (e.g. see the previous chapter).

→ Open Perception Studio's Acquire perspective

→ Set the length of the planned acquisition process (number of frames to be captured over time).

→ Click on Record button and wait till the system is ready for acquisition.

→ Confirm the start of the acquisition and move the measurement objects through the line of inspection

→ Crop the record to ensure reasonable data sizes

→ Provide documentation to this measurement

→ Save the record.

 The acquisition process

Acquisition of three plastic plates
An acquired hyperpsectral data scene of a hand holding three plastic plates. The outbounds of the scene were restricted to cover the object of interest. On the left, some description were attached to the data set.

Three plastics (VNIR).hsd

Step 3 - Explore hyperspectral data

Explore the acquired data to get an idea about its quality and e.g. about measurement errors.

→ Open Perception Studio's Explore perspective

→ Select the data on which exploratory investigation should get performed.

→ Inspect the scene

→ Select objects of interest and inspect its spectral information

→ Learn about the influence of different preprocessing

Step 4 - Model the application relevant information:

→ Open Perception Studio's Model perspective

→ Select the data for the modelling process.

→ Choose from one modelling method available in the ribbon.

→ Develop a model and save it for later usage.

Dependent on the application different approaches to modelling an information are available:

Model a colored perception (CCI)

Applying a CCI methods results in a chemical color image which expresses chemical information of observed objects by color information.

Model a classification

Applying a classification method results in a classification image which expresses a classification ID by a color information.

Model a statistical feature

Applying one of these methods results in a gray value image which expresses a statistical property by its pixel values.

Step 5 - Setup for live streaming:

In this step a CCI-compliant HW adapter is setup to perform a modelled feature extraction in real-time.

→ Open Perception Studio's Setup perspective and select the Perception Core to be the target device.

→ Make sure the hyperspectral system is set up properly.

→ Be sure the spectral range explained by your model fits to the spectral region of interest set.

→ Be sure the measurement system is calibrated. E.g. setup the balancing.

→ Scroll down the parameter list to the systems configuration panel and click on Add button.

→ Give the new configuration a meaningful name, select the models to be applied and configure streaming options.

→ Activate the new configuration by clicking on the Active button.

 Systems configuration panel

The configuration "Thress plastics" is available and activated.

Correlate_1stDerivative.psb

Step 6 - View live data: 

In this step the output of a CCI-compliant streaming adapter is viewed.

→ Open Perception Studio's View perspective and select the Perception Core to be the target device.

→ Activate the configuration of interest.

→ Click on Play button and inspect the live streaming.

 Live-view of a feature stream

The configuration "Three plastics" is activated. The model "Correlate_1stDerivative" get applied in real-time and results in a color stream shown in the view. Left, the three plastic plates got scanned in a crossed configuration (the plates overlap partly) - this results in a mixture of colors. Right, the three plastic plates got scanned without overlap.

Step 7 - Connect your machine vision application

This step summarizes the communication of your machine vision application with the streaming adapter (Perception Core).

→ Setup the Perception Core for live streaming (see a previous chapter).

• Streaming via UDP:
  Be sure to opt the stream type "UDP" when adding a new configuration to the Perception Core.
  Specify the IP and port your application is listen to.
• Streaming via GigEVision:
  Be sure to opt the stream type "GEV" when adding a new configuration to the Perception Core.
  Specify the NIC (streaming adapter) which should receive the streamed data.

→ Prepare your application to receive live data

• Streaming via UDP:
  Read the UDP Streaming Protocol Definition
• Streaming via GigEVision:
  Be sure your application can receive data from a GigEVision device.

→ E.g. prepare your application to control the Perception Core programmatically

• Read the UDP Control Protocol Definition

 GenICamBrowser program

The Perception Core is configured by the run job "Three plastics" (stream type = GigiEVision). Stemmer Imaging's GenICamBrowser program is used as client. On start of GenICamBrowser the device "Perception System" was selected. After pressing play, molecular information get extracted live from camera raw data and is shown in the program.

© 2019 by Perception Park GmbH
The content of this page and any attached files are confidential and intended solely for the addressee(s). Any publication, transmission or other use of the information by a person or entity other than the intended addressee is prohibited. If you receive this in error please contact Perception Park and delete copied material. Perception Park GmbH, Wartingergasse 42, A-8010 Graz; Austria; FN 400381x