In this tutorial the detection of plastic impurities on sausage is showcased


Outline

In food processing the detection of unwanted objects on goods is needed. This page summarizes a simple example on the detection of some plastic pieces on top of a stack of sausage slices.

The goal is to find a model which allows the distinction between the sausage (the good) and the plastic parts ( the unwanted objects). The obtained model is applied onto the measurement data and as a result a color image showing a distinction of objects through different colors is achieved.

In this tutorial a stack of sausage slices and plastics are investigated through NIR line-scan hyperspectral imaging. After a brief exploratory analysis, different CCI modelling methods are applied and the results are discussed.

Data, Samples & Measurement

Download the example project Sausage contaminated with plastics from the download section.


TitleSausage contaminated with plastics
CategoryFood - Impurity detection
Sample Description2x2 plastics on the top of a stack of sausage slices ("Extrawurst").
Sample IDunknown
Sample Locationunknown
DonorMarkus Burgstaller
Date of Measurement16-04-27
Measurement SystemSTEMMER IMAGING HS Setup: Allied Vision GoldenEye CL033SWIR, Specim N17E slit30um, KOWA F12.5, Perception System
Measurement DescriptionMeasurement of reflectance
Measurement SetupIllumination: Halogen diffuse, background: black foam
Measurement ID20160527132739
FormulaUnknown
AnalysisUnknown
PreprocessingUnknown
Spectroscopic DiscussionUnknown
Sample Image(s)





TitleSausage contaminated with plastics
CategoryFood - Impurity detection
Sample Description2x2 plastics on the top of a stack of sausage slices ("Extrawurst").
Sample IDunknown
Sample Locationunknown
DonorMarkus Burgstaller
Date of Measurement16-04-27
Measurement SystemSTEMMER IMAGING HS Setup: Allied Vision GoldenEye CL033SWIR, Specim N17E slit30um, KOWA F12.5, Perception System
Measurement DescriptionMeasurement of reflectance
Measurement SetupIllumination: Halogen diffuse, background: black foam
Measurement ID20160527132739
FormulaUnknown
AnalysisUnknown
PreprocessingTo enable highspeed scanning, the spectral range was restricted.
Spectroscopic DiscussionUnknown
Sample Image(s)



Start the project

Install the example project by double clicking on the downloaded file and following the instructions of the installer.

Open the Perception Studio program (e.g. from a link on your desktop) and change to the Start perspective.

Switch to example projects and select Sausage contaminated with plastics from the list shown in the Start perspective.

Now, all data are loaded into the Perception Studio program, please give the system some seconds time to load all the data.

Exploratory analysis of the data

Change to the Explore perspective and select the hyperspectral cube sausage_plastics from the project browser.
In the Explore perspective the loaded hyperspectral data (data set sausage_plastics) is visualized by an image in the center of the perspective as well as a spectra view on the bottom and a cube intersection view to the right.

Explore spectral information

The loaded example data comes along with some predefined spectra sets. Have a look at the spectra view and study the shown spectra sets.

The reflectance measurement was done in the NIR range (~1000-1700nm). The background spectrum (black foam) shows a reflectance around 0 over the whole spectra range. The plastics show distinct absorption bands at around 1150, 1200 and 1400nm. The reflectance spectra of the sausage shows a large influence of water (band at 1450nm). The spectra sets are showing varying reflectance information expressed by the area around the thick line.

It seems that both plastics are clearly distinguishable from the sausage in the spectral range of 1100-1250nm.

Explore the nature of the nested information

In the following image the Preview of the data is shown.

without1st derivative2nd derivative

In the image above the plastics are hardly distinguishable by their color information without preprocessing. By applying the 1st derivative the impurities are distinguishable from sausage by their color information. So, the spectra of the objects are much more distinct compared to the image without preprocessing. The 2nd derivative causes a distinction but of only small color contrast.

When the separation of plastics is desired, the "1st derivative" seems to be an appropriate candidate for preprocessing.

Explore statistical information

In the following illustration some statistical information about the reflectance data is shown.

MinMax

Mean

The objects of Plastic_2 get visible when investigating the minimum of the reflectance spectra - this is obvious since the minimum is quite different for spectra set Plastics_2 compared to all others (see section explore spectral information).
The maximum and mean of the reflectance spectra does not give a clear separation between impurities and sausage. The mean feature might give a stable criterium if the separation of objects from background is needed.

Model application relevant information

Change to the Model perspective and select the hyperspectral cube "sausage_plastics" from the project browser.

In the Model perspective the loaded hyperspectral data (data set sausage_plastics) is visualized by an image in the upper left of the perspective, a spectra view and a model view on the bottom left, as well as the result image and a control field on the right.

From the ribbon menu a set of CCI modelling methods are accessible. An obtained model is saved in the project by clicking on the Save Model button and is later accessible from the project browser to the left of the Model perspective.

Modelling based on the extraction of principle components

From pre-selected spectra sets (sausage, plastic_1, plastic_2, etc.) principle components are extracted automatically. Their score per pixel is shown in the image collection in the bottom right.

Please note that by applying the Extract method spectra, get "unscrambled" into components. Typically a handful of components are necessary to describe the major part of information nested in the spectra.

Doing so, the components which potentially hold application relevant information are selected. If no application relevant information is available from the score images, the Extract method might not be suitable for your application. In this case, continue with one of the other CCI methods available (e.g. Correlate or Constrain).

Without preprocessing

Set Intensity in the Preprocessing section and select the CCI modelling method Extract.
In the following, unscrambled component score images are shown:


PC1


PC2


PC3


PC4


PC5

PC6

PC1, PC2 and PC3 hold information valuable for solving the application - they distinguish the plastics from the stack of sausage slices.
PC4 holds information about the sausage (the spots might correspond to varying fat content).
PC5 and PC6 primarily show information about a detail of plastic_2 (also part of PC1 and PC2).

To influence the unscrambling process of principle components, you can arrange the spectra selection differently (e.g. reject the background spectra if you only want the objects to influence the unscrambling process) or you should consider to apply one of the Preprocessing methods, accessible from the ribbon menu.

A perception (a color image) is gained by assigning appropriate PC's to the color channels of the resulting color image.
In this example the assignment is chosen automatically to be:

  • PC1 >> red
  • PC2 >> green
  • PC3 >> blue

This assignment results in the following image shown in the upper right of the Model perspective:

Resulting perception (PC1 => red, PC2 => green, PC3 => blue)

plastic_1 is described by an average value of PC1 (red), a low value of PC2 (green) and a high value of PC3 (blue). Therefore, plastic_1 is shown by a violet-ish color in the perception.
plastic_2 is described by a high value of PC1 (red) and PC2 (green) but a small value of PC3 (blue). Therefore, plastic_2 is shown by a yellow-ish color in the perception.
Since PC1 (red) is the highest value of the sausage, the sausage is shown by a red-ish color.

By assigning other score images to the perceptions color channels, other perceptions can be obtained. In the following example PC4 (which describes a varying information, most likely fat content in the sausage) is assigned to red, green and blue - therefore a gray value image is obtained. The varying information in the sausage becomes visible:

Resulting perception (PC4 => red, green, blue)

Apply preprocessing to spectra

In the next section we would like to investigate preprocessed spectra in combination with the Extract method. In section explore spectral information we identified the spectral information of sausage and plastics to be much more distinct when 1st derivative is selected as Preprocessing from the ribbon menu.

Please note that the unscrambling of spectra into components is yielding better results, when the spectra are preprocessed properly.

Select sausage_plastics data set from the project browser and select 1st derivative in the Preprocessing section of the ribbon menu.
This results in the following component score images:


PC1


PC2

PC3

PC4

PC5

PC6

PC1, PC2 and PC3 hold information valuable for solving the application - they distinguish the plastics from the stack of sausage slices.
PC4, PC5 and PC6 hold information about noise.

The assignment of PC's to the result image is shown in the following image (upper right of the Model perspective):

Resulting perception (PC1 => red, PC2 => green, PC3 => blue)

We can summarize, that by applying the 1st derivative, only 3 components are needed to describe the spectral information in the data. By assignment of this PCs to color channels a perception is gained, which describes the distinction between sausage and plastics very well.

We suggest to apply preprocessing to the data even if the obtained results without preprocessing seem to be comparable to the ones with preprocessing.
Preprocessing typically allows applications to be more independent from environmental influences (like varying lighting etc.). So such models provide typically more stable results compared to models without preprocessed spectra.

Summary

By means of the Extract method, a perception was obtained capable of detecting unwanted objects (plastics) on a good object (sausage). The information extraction was done unsupervised (the method automatically unscrambles selected spectra into principle components). The unscrambling result can be greatly influenced by preprocessing of spectra or by the selection of spectra.

The transported information is selectable by the user through the assignment of score images to the color channels of the resulting image.

We suggest to use the preprocessing 1st derivative instead of no preprocessing - such a model is typically much more stable against environmental influences - it obtains more robust results in an industrial environment.

Modelling based on the correlation of spectra sets

Pre-selected spectra sets (sausage, plastic_1, plastic_2, etc.) are interpreted by the software as pure components and are correlated to the scene. Their correlation per pixel is shown in the image collection in the bottom right of the Model perspective.

Please note that a spectrum can be a mix of different spectra of pure components. By knowledge of  „all“ pure components, a correlation value per component and pixel can be calculated. By assignment of component score images to colour channels, a perception is obtained.

Without preprocessing

Set Intensity in the Preprocessing section and select the CCI modelling method Correlate.
In the following correlation score images are shown:

SausagePlastic_1Plastic_2background


PC1, PC2 and PC3 hold information valuable for solving the application - distinguish the plastics from the stack of sausage slices.
PC4 holds information about the sausage (the spots might correspond to the varying fat content).
PC5 and PC6 primarily show information about a detail of plastic_2 (also part of PC1 and PC2).

Make sure to only select objects which can be understood to be "pure" - spectroscopically independent from each other (orthogonal).
In case you miss this prerequisite, the results might not be of value to you - they might get noisy.

To potentially improve the correlation process, you should consider applying one of the Preprocessing methods accessible from the ribbon menu.

A perception (a color image) is obtained by assigning appropriate score images to the color channels of the resulting color image.
In this example the assignment is chosen automatically to be:

  • sausage >> red
  • plastic_1 >> green
  • plastic_2 >> blue

This assignment results in the following image shown in the upper right of the Model perspective:

Resulting perception (sausage => red, plastic_1 => green, plastic_2 => blue)

Applying preprocessing to spectra

In the next we want like to investigate preprocessed spectra by the Correlate method. In the section explore spectral information we recognized the spectral information of sausage and plastics to be much more distinct when 1st derivative is selected as Preprocessing from the ribbon menu.

Please note that in general, the correlation of pure components is yielding better results, when the spectra are preprocessed properly.

Select sausage_plastics data set from the project browser and select 1st derivative in the Preprocessing section of the ribbon menu and study the results. After that, select 2nd derivative and try to understand the differences.

This results in the following components score images:


SausagePlastic_1Plastic_2background
1st der
2nd der

The obtained correlation score images of preprocessing 1st derivative are quite similar to those without preprocessing.
This is different to the results obtained by applying the 2nd derivative. The 2nd derivative shows a lot of noise in the spectra - therefore the score images are noisy too.

The following images summarize perceptions gained when different preprocessing is applied.

without1st der2nd der

Resulting perceptions (sausage => red, plastic_1 => green, plastic_2 => blue)

We can summarize, that by applying the 1st derivative, the results are quite similar to the results without preprocessing. The 2nd derivative tends to produce noisy results.

We suggest to apply preprocessing to the data even if the obtained results without preprocessing seem to be comparable to the ones with preprocessing.
Preprocessing typically allows applications to be more independent from environmental influences (like varying lighting etc.). So such models provide typically more stable results compared to models without preprocessed spectra.

In case a higher order derivative has to applied, one should consider to apply some smoothing on the gained output image in order to reduce noise.

Unsuitable selection

Now, we would like to do an unsuitable selection to see how results can change.
Set 1st Derivative in the Preprocessing section and select the CCI modelling method Correlate.
Open the Select tool from the Edit section in the ribbon and append a new spectrum:

Click on + New Spectra Set button and select an object which is already defined by an available spectra set.
I selected some pixels (red dots) of the 2nd piece of plastic_1 and named it plastic_1b:

You can modify the name and color of spectra sets by clicking on its name or color box respectively.


In the following correlation score images are shown:

sausageplastic_1aplastic_1bplastic_2background

The gained correlation score image are now unsuitable. It is not possible to correlate spectra to the scene if they are not orthogonal to each other (does not have something in common with each other). This gets visible by studying the results for plastic_1a and plastic_1b which describe the same chemistry and result in noisy images.

Summary

By means of the Correlate method, a perception was obtained capable of detecting unwanted objects (plastics) on a good object (sausage). The information extraction was done unsupervised (the method automatically correlated selected spectra to the scene). The correlation process can be greatly influenced by an unsuitable selection of spectra. Be sure to select spectra which hold independent information (orthogonal information), so pure components (constituents).

The transported information is selectable by the user through the assignment of score images to the color channels of the resulting image.

We suggest to use the preprocessing 1st derivative instead of no preprocessing - such a model is typically much more stable against environmental influences - it obtains more robust results in industrial environment.

 


 © 2019 by Perception Park GmbH
The content of this page and any attached files are confidential and intended solely for the addressee(s). Any publication, transmission or other use of the information by a person or entity other than the intended addressee is prohibited. If you receive this in error please contact Perception Park and delete copied material. Perception Park GmbH, Wartingergasse 42, A-8010 Graz; Austria; FN 400381x