The Model perspective of the Perception Studio program provides tools for the creation of models, which can be applied to hyperspectral information.

Motivation

A model is a simulation of a system to reproduce its behavior for a defined input range.
Once a model is designed, it can be applied to new data (of same system). This results in a value used as scale of the behavior that was modeled.

By applying a sugar-model onto one spectra, the sugar content represented by the spectra is measured - it results in 1 value, the sugar content"
By applying a sugar-model to hyperspectral data, a monochrome image is obtained depicting the sugar content by its pixel values (gray values).

Please keep "qualitative" and "quantitative" investigations separated. Models for quantitative measurement are designed by means of chemometric calibration routines
while qualitative models are obtained by applying the CCI-Methods described below.

By applying a CCI-model to hyperspectral data, a perception of qualitative information is obtained. The information format is color. The color describes the behavior that was modeled.

A model itself is a structure of values and is typically hard to interpret without specific spectroscopic knowledge.

Layout of the Model Perspective 

In the Model perspective of the Perception Studio program you can

• edit hyperspectral data,
• select spectra sets,
• preprocess the hyperspectral data,
• develop new models
• apply existing models to hyperspectral data
• edit existing models
• save a model for later use

The main panels of the Model perspective are

1. Model Selection (create, rename, delete, duplicate or select models)
2. Method Selection (select model type)
3. Preprocessing Options (set preprocessing for the current model)
4. Training Data Panel (select training data and edit it)
5. Test Data Panel (select data to test the model)
6. Selected Spectra Plot (display spectra sets of interest)
7. Method Editor (edit the selected model)
8. Feature Model Graph (see the current model curves)
9. Info Panel (shows information)

1 Model Selection 

The model selection shows the currently selected model and its name. When clicked, a dropdown box appears, which shows all models that are contained in the currently loaded project as well as buttons to

• create a new model
• import a model from file
• duplicate the selected model
• rename the selected model
• export the selected model
• delete the selected model

Here is an example:

2 Method Selection 

At the moment, there are 3 categories of modelling methods as described below. When a model is selected, The corresponding method button will be highlighted to identify the type of the model. The method of an existing model can be changed. This will relearn the model and is only possible if the model has valid Training Data.

CCI (Model a colored perception)

The CCI (Chemical Colour Imaging) methods section in the ribbon menu enables you to select from following CCI methods:

• CCI Preview Method: unsupervised method showing spectroscopic differences in different colors.
• CCI Extract Method: extraction based on variance information.
• CCI Correlate Method: correlation of pure substances
• CCI Constrain Method: constraining colors
• CCI Design Method: design of CCI-method model

Applying one of these methods results in a chemical color image which expresses chemical information of observed objects as color information.
After a CCI-model is developed the model has to be saved for later usage.

Classification

The classification methods section in the ribbon menu enables you to select from following classification methods:

• Case Discrimination: 1-Click classification method enabling the discrimination of different cases (e.g. objects).

Applying one of these methods results in a classification image which expresses a classification ID by a color information.

After a classification-model is developed the model has to be saved for later usage.

Features

The feature methods section in the ribbon menu enables you to select from following feature methods:

• TrueColor extraction: Provides TrueColor measurements if the data covers the visual range. 
• Statistical Features: The value per pixel is defined by a statistical function.
  Applying one of these methods results in a gray value image which expresses a statistical property by its pixel values.

After a feature-model is developed the model has to be saved for later usage.

3 Preprocessing Options 

Select from a list of preprocessing methods by clicking on the associated thumbnail in the ribbon menu and use the dropdown box to apply normalization.

Preprocessing is performed along the spectral domain.

4 Training Data Panel 

The training data panel shows the current training data. In the top area, one can select a training data set from the data sets in the current project as well as import, export, rename and delete data sets in the project. 

In the top right of the training data panel there are several buttons to edit the current training data set as follows in the next subsections

Crop a Hyperspectral Cube

See documentation: Crop Hyperspectral Data

Click on Crop button in the Edit section of the ribbon menu to crop selected hyperspectral cube data. When done, a new cube is generated and is available from the project browser.

Select Spectra

See documentation: Select Spectra

Click on Select Spectra button in the Edit section of the ribbon menu to start the selection process. The modelling process is based on the selected spectra.

Merge Hyperspectral Cubes

See documentation: Merge Hyperspectral Data 

Click on Merge button in the Edit section of the ribbon menu to merge cubes (to get one cube comprised of individual cubes). When done, a new cube is generated and is available from the project browser. 

5 Test Data Panel 

The test data panel allows the user to select a data set to test the currently selected model on. The dropdown selection at the top allows the selection of a data set from the current project, as well as importing, exporting and renaming data sets. When the current model is applied to the test data set, the output image is displayed.

Errors are shown in this panel if

• the spectral resolution of the model and the test data set do not match
• there is no model selected
• the selected model is invalid because it failed to calculate from the given test data
  
  

6 Selected Spectra Plot 

The selected spectra plot shows the selected spectra of the training data set and thus may help to identify differences in materials. If the mouse is hovered above the training data image above, the spectra plot also displays the spectrum under the cursor.

For information on how to select spectra, see documentation: Select Spectra

7 Method Editor 

The method editor  shows all parameters of the currently selected model. Here, the model can be changed and fine tuned. The UI elements, shown in the model editor vary, depending on the selected method type (top bar). An existing model can only be edited, if a valid Training Data is available for the model. If this is not the case, an information will be shown, which tells the user, which data set is expected. When loading this exact data set into the project, the model will be editable again.

For information on the different editors for all available methods, see Method Selection

8 Feature Model Graph 

The Feature Model graph shows information about the model itself, such as the wavelength range, that the model can be used for. Note, that this is not available for all method types.

9 Info Panel 

The info panel shows information about the selected data set or the selected model. When moving the mouse over other panels, such as the input view, output view, selected spectra or feature model, information about the shown image/plot is displayed. This information includes information like feature values, color values (per pixel), wavelength, etc.

Development of a model

To develop a model, used to extract an application relevant information (such as a molecular information resolved by pixels), be sure to have a hyperspectral cube available within the project browser.

Acquire a hyperspectral cube by means of a hyperspectral camera interface to the Perception Studio program or load example data available from the Perception Wiki.

Step-by-step guide:

1. Switch to the Model perspective to start your development.
   
   
2. Select a hyperspectral cube from the Training Data Panel.
   
   

3. In the Training Data Panel you should see the Preview of the selected hyperspectral data

   Please refer to the Preview Method documentation for further information.

   
   

4. Consider cropping the data to get a reasonable data size.
   Concentrate only on objects of interest, reject environment information which does not carry valuable information. By this, you can greatly speed up the processing.

   Please refer to the Crop Hyperspectral Data documentation for further information.

   
   

5. Select objects of interest.
   Depending on your strategy, select pixels from objects in the scene which seem to be important to you. Give them a meaningful name and color.

   Please refer to the Select Spectra documentation for further information.

   
   

6. Create a new model by either 

   1. Selecting a model method from the top ribbon bar, e.g. select the CCI-Extract method shown in various images on this page.

   2. Press "New Item" in the Model Selection dropdown menu.
   3. Press "Create new model" in the Test Data Panel if this hint is shown.

7. Parametrize the method from the control panel located on the bottom right in the model perspective.

   Please refer to the model methods documentation for further information

   
   

8. Study the result in the Test Data Panel.
   To improve the quality of the result vary your spectra selection, adapt the parametrization of the model method, try another model method and/or apply preprocessing to the hyperspectral data.

   Please refer to the Strategies on the Configuration of Hyperspectral Preprocessing documentation for further information.

   
   

9. Apply the model to new hyperspectral data.
   Setup a new run-job, so the developed model is applied live to new camera data.

   For further information please refer to the Setup a Hardware Device documentation.

   Or apply the developed model to new hyperspectral data.

   For further information please refer to the next section in this documentation.

   
   

Applying an existent model to new hyperspectral data

Often when work offline, a model (developed in past) has to be applied to new hyperspectral data (e.g. acquired a moment ago).

To do so make sure that the model and the hyperspectral data are accessible from the project browser (e.g. import the data from file).

• Switch to the model perspective.
• Select the hyperspectral cube in the Test Data Panel.

Now, the Test Data Panel in the model perspective will show the result of the selected model applied to the selected hyperspectral data or it will display an info message if the model can not be applied to the selected data set

© 2019 by Perception Park GmbH
The content of this page and any attached files are confidential and intended solely for the addressee(s). Any publication, transmission or other use of the information by a person or entity other than the intended addressee is prohibited. If you receive this in error please contact Perception Park and delete copied material. Perception Park GmbH, Wartingergasse 42, A-8010 Graz; Austria; FN 400381x