Introduction
The rich nutritional profile (vitamin A, D, folic acid, niacin and biotin; phosphorus, sodium, zinc, magnesium, calcium and iron; higher protein and lower fat content) of mushrooms helps it to be placed in the healthier and nutritious food category (Bains et al., 2021; Jardim-Botelho et al., 2022; Lu et al., 2021). Due to active bio-components, mushrooms have antimicrobial, antitumour, haematological and antioxidant activities (You et al., 2022). According to Müller and Krawinkel (2005) it is considered as a food that may help to combat malnutrition along with the challenges such as health, hunger and poverty (millennium development goal for third world countries.
The food sensory evaluation and studies are processes of assembling information used to quantify, investigate and infer the human behavioural reactions towards some food material in terms of touch, taste, sight, smell and hearing; the panelists are working as a measuring instrument to quantify the responses to ascertain the quality of the food (Mukherjee et al., 2022). The sensory analysis is encompassed with the sequential array of methods considered to monitor human perception (Choi et al., 2020). The characteristics and quality of the food product studied can be judged from the sensory study and may be utilised in consumer understanding, development of the new product, quality control and taste profiling (Lee et al., 2021; Yu et al., 2018).
The consumer preference or acceptance of a food product is quantified with the numerical values in hedonic testing (Mukherjee et al., 2022). The assessor panel produces sensory scores against sensory attributes of food products which are therefore termed as sensory data. Sensory attributes are reflected in the textural, taste, smell and other physicochemical properties. Other extrinsic properties such as nutritional, price and branding information correlate to sensory attributes. Sensory evaluation is directly related to the consumer acceptance of a product; thus, according to the result of hedonic testing, improvement of a food product may be made (Chaari et al., 2022a; Fourati et al., 2020; Kiran et al., 2022; Sarkar et al., 2020).
A large number of variables are generally associated with sensory analysis. Thus there are some problems such as (a) the training panels require more time to recognise these huge attributes. It becomes costly; (b) while working with a higher number of variables, the discriminant methods need to perform better. The performance of these methods can be enhanced if lesser variables are used (Granitto et al., 2007). For projection methods such as PCA, selecting the first components produces better results and performance than the discriminant methods. However, the panel is trained to work with the full attributes.
The artificial neural network (ANN) has extensive use in finding a solution in such a situation when it is difficult to apply other statistical methods. The characteristics of this method, such as forecasting non--linear data fault tolerance, the ability to learn from examples and operation in a real-time environment, are the key advantages associated with this method (Chaari et al., 2022b; Xu et al., 2022). The basis of this technique is the features and biological neurons of human brain. The concept of the function of the neuron is used here mathematically. Generally, a layering arrangement exists between the artificial neurons, namely the input (consisting of input dataset), hidden, and finally, the output (final result) layers. The neurons that belong to the same layer are not connected although the neurons of consecutive layers are interconnected. The nodes adjust themselves to achieve the solution with the highest competency depending on the weighted connections. A robust network can be built by minimising errors and updating weighted attributes -simultaneously (Ennouri et al., 2021; Lahiri et al., 2021; Singh et al., 2022).
In the sensory analysis of food materials, the multivariate projection techniques, namely generalised Procrustes Analysis (GPA), discriminant analysis (DA) and principal component analysis (PCA), are considered in order to differentiate the food products (Rossini et al., 2012). Researchers prefer linear DA to diagnose smoked products (Ojeda et al., 2022), while ANN has been used to differentiate Scotch whiskey varieties based on its features (Jack and Steele, 2002). In the recent era, machine learning techniques are conquering the challenges of the food industries in terms of developing predictive models for determining food quality (Irfan et al., 2022; Martínez-Simarro and Lázaro-Ramos, 2022). ANN has been efficiently applied in the shelf life and quality prediction of versatile food materials, such as processed cheese (Stangierski et al., 2019) and fish (Liu et al., 2015). The temperature-dependent kinetic model (Wang et al., 2012) or statistical models (Pleșoianu and Nour, 2022) are considered to construct the shelf-life predictive model for Shiitake and Matsutake mushrooms, respectively (Fu et al., 2019).
This study captured digital images of oyster mushroom samples with smartphone devices. Features were extracted from the images to construct a robust model to predict sensory attributes such as shape, colour and texture to exclude expert dependency.
Materials and Methods
Sample collection
The fresh (harvested and purchased on the same day) oyster mushrooms (Pleurotus florida) samples were procured (non-appearance of foreign body and insect damage as appeared under naked eye) from ISO 9000/IS 14000 certified agricultural co-operative of Malda, West Bengal, India. The experiment was started immediately after the procurement. The ambient temperature and relative humidity of the laboratory were relatively sustained at 25 ± 5°C and 80 ± 5%, respectively. To allow the natural decomposition of the samples, no preservatives were added, and the samples were kept away from direct sunlight.
Image capturing
Each sample was kept on an A4 size white paper, and images were captured with Galaxy M31 (Samsung, South Korea) smartphone device in natural daylight for 5 consecutive days (Figure 1). In total, 135 mushroom samples were considered, and images were captured once per day. Therefore 675 images were captured and used for the study. The technical specifications and details of the image capture procedure were in line with our previously published work (Mukherjee et al., 2022).
Figure 1. Different mushroom sample images.
Panel preparation for evaluation of mushroom quality
In total, 129 people (non-smokers and healthy; 81 males and 48 females) were assessed to form the panel to evaluate the quality of the mushroom samples. Initially, a triangle test was conducted to screen the panelists as per ASTM E1885-04, 2011 (Mukhopadhyay et al., 2013). Ultimately, a 90-member panel (57 males and 33 females with a 70% success rate in the triangle test) was built to evaluate mushroom quality. The panelists were trained about the scoring method, attributes selected to judge the quality through visual inspection and the features of good and bad quality mushroom samples as per ASTM STP-758, 1981 and ASTM MNL-26, 1996 method.
Hedonic scale rating
The assessors (regular basis consumers of oyster mushrooms) considered three specific quality characteristics that can be assessed through eye estimation, namely the shape, texture and colour of the samples. They rated the quality characteristics of oyster mushrooms based on a 9-point hedonic scale, where 9 = “liked extremely,” 8 = “like very much,” 7 = “like moderately,” 6 = like slightly,” 5 = “neither like nor dislike,” 4 = “dislike slightly,” 3 = “dislike moderately,” 2 = “dislike very much” and 1= “dislike extremely.”
Pre-processing of the images
The digital images captured were cropped along the centre part (region of interest) to accommodate the highest possible area covered by the mushroom sample and exclusion of the background. Further, the images were resized to the dimension of 300 × 300 to make the data matrix uniform.
Feature selection and feature extraction
Using ANN modeling, 11 major features (10 colours and 1 texture) were selected to predict the sensory attributes through visual inspection. The digital images of mushrooms were considered with three different colour maps, and those were a) Red (R)-Green (G)-Blue (B) colour map; b) Luminance (Y)-Chrominance (Cb and Cr) colour map; c) Hue (H)-Saturation (S)-Value components (V) colour map. All three colour maps consist of three segregated layers. The default colour map for digital images captured was the RGB layer; thus, the separation of the independent colour channels provides the images with three separate (R, G and B) colour space. The intensity histogram (IH) was developed for each colour channel, and the IHs were used as the input variable for ANN models. After that, the default colour space (RGB) was converted to YCbCr and HSV colour space. Similarly, IH (in pixel level) was extracted for images in the colour maps mentioned above. The images were transformed to the greyscale to obtain the IH in greyscale. The major texture feature, entropy, was analysed for the images to find out the changes in texture level. The datasets were built in a 256 (pixel-level consideration) × 675 (numbers of images) matrix for all the 11 features and considered for the later part of the experiment.
Artificial neural network model
The ANN model was built with MATLAB R2014b (MathWorks Inc., USA). The dataset was in the ratio 70:15:15 (training: testing: validation). The scaled conjugate gradient backpropagation (SCGBP) algorithm (for hidden layer, the activation function was Sigmoid; and for output layer, ReLu activation function was considered) was considered to construct the model (Sarkar et al., 2021). The model comprised 256 inputs, 10 hidden and 1 output layer (learning rate 0.5–0.9) (Figure 2). In total, 11 features of the digital images were considered as input variables, while the output features were shape, texture and colour. The ANN model was built to find the best-performing feature that can predict consumer perception based on sensory quality (in terms of shape, texture and colour) from the smartphone-based image analysis. The proposed work is depicted with the help of Figure 3.
Figure 2. The proposed ANN architecture.
Figure 3. Flowchart of the proposed methodology for digital image analysis aided sensory analysis of mushrooms.
Results and Discussion
Results
The step size (for SCGBP) at the nth iteration was determined using Equation (1).
Where, GE = function of global error; x = weight vector; S = scalar parameter that adjust itself to s; v = non-zero vectors; d = distance
Upon adjustment, the step size becomes
If the comparison parameter (CPn) ≥ 0.75, the sn becomes
If CPn ≤ 0.25, the sn becomes
The training cycle stops upon attainment of the following conditions (Maiti and Tiwari, 2010):
-
The numbers of epochs attain maximum value (we have set this at 1000).
-
The time attains a maximum value
-
The minimum performance value is reached
-
The minimum-gradient value is reached by the factor, namely the performance gradient
-
The performance of the validation cycle becomes higher than the previous maximum fail time.
The results of correlation coefficients between the true Hedonic level and those of the predicted Hedonic numbers, obtained from the proposed ANN model regarding the colour, shape and texture quality prediction of the mushroom samples are described in detail in Tables 1, 2 and 3, respectively.
Table 1. The coefficient of correlation values for the shape prediction models.
| Image features | Training | Testing | Validation | All |
|---|---|---|---|---|
| Red channel | 0.86060 | 0.85500 | 0.77658 | 0.84672 |
| Green channel | 0.88854 | 0.93878 | 0.80533 | 0.86887 |
| Blue channel | 0.82605 | 0.89409 | 0.81918 | 0.83583 |
| Hue | 0.87653 | 0.91142 | 0.74286 | 0.85654 |
| Saturation | 0.88634 | 0.86854 | 0.82843 | 0.87923 |
| Value components | 0.84142 | 0.87745 | 0.88370 | 0.85594 |
| Luminance | 0.85775 | 0.86160 | 0.84955 | 0.85151 |
| Chrominance (Cb) | 0.76670 | 0.76132 | 0.61883 | 0.74271 |
| Chrominance (Cr) | 0.92202 | 0.87890 | 0.84879 | 0.90721 |
| Entropy | 0.80204 | 0.86428 | 0.82347 | 0.80810 |
| Greyscale | 0.84271 | 0.90667 | 0.92109 | 0.86403 |
Table 2. The coefficient of correlation values for the texture prediction models.
| Image features | Training | Testing | Validation | All |
|---|---|---|---|---|
| Red channel | 0.87692 | 0.85704 | 0.89302 | 0.87132 |
| Green channel | 0.84441 | 0.88961 | 0.82337 | 0.84971 |
| Blue channel | 0.87011 | 0.96501 | 0.85820 | 0.86427 |
| Hue | 0.83608 | 0.85761 | 0.86651 | 0.84207 |
| Saturation | 0.84133 | 0.90050 | 0.82287 | 0.84840 |
| Value components | 0.80974 | 0.89369 | 0.84548 | 0.81869 |
| Luminance | 0.89826 | 0.84773 | 0.85555 | 0.88095 |
| Chrominance (Cb) | 0.86477 | 0.76218 | 0.72120 | 0.82786 |
| Chrominance (Cr) | 0.85257 | 0.86778 | 0.83763 | 0.85382 |
| Entropy | 0.82189 | 0.81115 | 0.93451 | 0.84059 |
| Greyscale | 0.83666 | 0.89270 | 0.84181 | 0.84351 |
Table 3. The coefficient of correlation values for the colour prediction models.
| Image features | Training | Testing | Validation | All |
|---|---|---|---|---|
| Red channel | 0.87117 | 0.93631 | 0.85120 | 0.87164 |
| Green channel | 0.89538 | 0.90758 | 0.87346 | 0.89025 |
| Blue channel | 0.91754 | 0.89913 | 0.90231 | 0.91180 |
| Hue | 0.85834 | 0.86426 | 0.89653 | 0.86974 |
| Saturation | 0.90391 | 0.93315 | 0.93391 | 0.91227 |
| Value components | 0.84338 | 0.89418 | 0.84501 | 0.85413 |
| Luminance | 0.91383 | 0.88587 | 0.88915 | 0.90144 |
| Chrominance (Cb) | 0.81405 | 0.79865 | 0.88785 | 0.82456 |
| Chrominance (Cr) | 0.90297 | 0.88875 | 0.91408 | 0.89807 |
| Entropy | 0.85217 | 0.87071 | 0.92013 | 0.86342 |
| Greyscale | 0.93091 | 0.89096 | 0.83915 | 0.90732 |
Performance analysis of the proposed ANN model
The following statistical indices were considered for the analysis of the performance of the proposed ANN model:
Where predicted = predicted value with ANN model; actual = values derived from image analysis; MA = mean of actual; x = 256 (numbers of observations).
The MAE and NAE values closer to 0 depict a more robust model, while the higher I and AP values (closer to 1) represent the better performance of the prediction model. At the same time, I signifies how closely the model--predicted data can simulate the actual data obtained from the image analysis. The magnitude of the errors, irrespective of their direction, is measured with the MAE. In this study, out of 33 models, the model built with saturation IH as input and colour (in terms of hedonic values) attribute as the output showed minimum MAE (0.033) and NAE (0.105) values and maximum I (0.963) and AP (0.954) values. The fitness plot for the prediction model for a colour attribute is in coherence with the result perceived (Figure 4). Similarly, the model built with chrominance (Cr) IH as input to predict the shape (in terms of hedonic values) attribute showed minimum MAE (0.042) and NAE (0.126) values and maximum I (0.945) and AP (0.939) values. For the texture (in terms of hedonic values) prediction model, the model built with the luminance feature was the best-performing one with the minimum MAE (0.072) and NAE (0.150) values and maximum I (0.921) and AP (0.910) values.
Figure 4. The fitness plot of the colour (in terms of hedonic values) prediction model built with saturation values.
Analysis of the overall results
The overall analysis of the results regarding the colour, shape and texture quality prediction of the mushroom samples, as obtained from Tables 1, 2 and 3, are shown graphically as a bar plot in Figure 5.
Figure 5. Bar diagram of the correlation coefficients between the true Hedonic level and the overall predicted Hedonic level considering the colour, shape and texture of the mushroom samples using ten colour component features and one entropy feature.
It is found from Figure 5 that colour quality prediction is obtained best for most of the features, especially with blue, saturation, luminance, Chrominance (Cr) and greyscale features, where the correlation coefficients are found to exceed 0.9. The shape prediction has lesser accuracy, with only the chrominance (Cr) feature having a correlation coefficient higher than 0.9. The correlation coefficient of predicting the same parameter has even gone to less than 0.75 with chrominance (Cb) feature. It is further found that none of the features has been able to exceed the correlation coefficient of 0.9. Further, Figure 5 shows the mean of the correlation coefficients between the true and predicted Hedonic levels, which shows that the best mean prediction is obtained with the chrominance (Cr) feature, considering the mean of the predicted colour, shape and texture parameters.
Thus, it is observed from Figure 5 for most of the component features that the predicted Hedonic number is closest to the true value for the colour level prediction, as these have the highest correlation coefficients in most cases compared to the shape and texture parameters. In order to affirm the same, we have further plotted these values using best-fit linear models in Figure 6. The three parameters, that is, colour, shape and texture of the samples, are represented using best-fit linear models using the correlation coefficients between the true and predicted Hedonic levels with all the eleven image features.
Figure 6. Bar diagram of the mean correlation coefficients between the true Hedonic level and the overall predicted Hedonic level considering all three features, such as colour, shape and texture of the mushroom samples, using ten colour component features and one entropy feature.
It is easily found from Figure 7 that the best-fit line considering the correlation coefficients of the colour parameters, is lying much above the other two best-fit lines, which are formed using the correlation coefficients of the shape and texture parameters. Again, it is observed that the prediction models with shape and texture parameters also lie very close to each other; hence, the prediction levels are very much identical. The average correlation coefficient level with the colour parameter is close to just above 0.88, whereas the other two parameters lie approximately between 0.84 and 0.86. Hence, it is affirmed here that the colour quality prediction is obtained best for the colour of the samples.
Figure 7. Comparative analysis of the prediction of colour, shape and texture parameters.
Discussion
Analysis of the results from Tables 1, 2 and 3 indicates the effectiveness of the proposed scheme in efficiently judging the quality of the mushroom samples, as indicated by the predicted Hedonic number of the ANN model, as it is observed that the samples degrade progressively with days of progression. It is observed that the whitish mushroom images degrade to a brownish colour over days. It is qualitatively inferred that the histogram profiles of the RGB layers would change noticeably as the samples move from fresh quality to deteriorated form. Hence, we have analysed these colour histogram features to identify the varying colour features. Another vital colour map, the HSV colour map, has also been analysed. The hue and saturation-based colour map are less influenced by the luminosity of the images, which makes the image analysis more robust in terms of invariance of the luminosity of the image-capturing environment. Thus, we analysed these colour features. Apart from that, we have also analysed the greyscale colour map of the samples. Further, we have found that the texture of the mushroom samples deteriorates significantly with time. Hence, we also analysed the textural features of the samples by using the entropy filtering of the image, thereby developing an entropy-filtered image from the greyscale colour map. The histogram map of the entropy-filtered image was also analysed using the same model. Therefore, the proposed work comprises an analysis of the samples’ colour, shape and texture features, which are the major features of a similar analysis.
The proposed method of estimation of mushroom quality bears major significance, both for industrial automation and for the individual and isolated purchase of mushrooms by the common people. Assessment of the quality of any food product is essential. As discussed, the Hedonic scale indicates the quality of the samples in terms of all three major quality assessment parameters, such as the samples’ colour, texture and shape. In order to judge the quality of any fruit or vegetable sample using the Hedonic scale, we need a panel of experts who would judge the three major features of the samples using a number scale between 0 and 10, depending on the quality of the sample. However, it is difficult, under any condition, to arrange such a good number of experts, all at a time, for such quantitative judgment of the quality of samples. This is more important under the Covid scenario when accumulating people at one point becomes much difficult. Thus, a close approximate result regarding the quality of the samples would be more beneficial for the buyers or the “v” automation industries to assess the quality of the samples “m,” even in the absence of a team of experts.
Apart from that, the proposed model gives a direct and continuous assessment of the quality of the mushroom images instead of a discrete classification of the samples into two or three classes to judge the edibility or non--edibility of the same. Hence, the model gives a fair prediction of the level of consumability and the extent of degradation of a test sample in terms of the predicted Hedonic number. Thus, the proposed model does not declare if a sample is consumable; rather, it estimates how far it has deteriorated in quality by predicting the Hedonic level in all three output parameters of the sample, that is, its colour, shape and surface texture. Thus, a person or an automation system with a fair idea of these parameters could easily judge whether to purchase, consume or process the same.
The proposed supervised learning-based algorithm is found to yield high efficiency in assessing the samples’ quality. This is inferred from the high correlation between the true Hedonic number marked by the team of experts, and the algorithm-predicted Hedonic number, regarding the three basic quality features such as colour, texture and shape variation of the samples.
The above three features of the proposed algorithm regarding the high accuracy of quality prediction and non-requirement of experts make the same effective. Apart from these vital features of this work, another few features have also been instrumental in this context, as given below:
-
The proposed model uses only a single and simplified form of ANN for developing the proposed algorithm. This ease in computation also makes the scheme practically impenetrable. Besides, the computation time is also on the lower level due to the less intense complexity of analysis. These features are extremely useful for implementing the algorithm in low-memory application software.
-
Another vital aspect of the work is that the images are all captured using smartphone-inbuilt cameras. Smartphones are almost a usual device nowadays, even for the common people. The ease in application of the algorithm, simplicity of analysis, high accuracy of quality prediction of the mushroom samples in terms of the Hedonic number, especially in the two most important quality aspects such as colour and texture, and analysis of the sample images captured using smartphones only make this work implementable for developing smartphone-based application. The possible development of such a smartphone-inbuilt application would widen the applicability of the proposed algorithm.
-
In real-time marketplace scenario, taste and smell are difficult to measure; and most commonly, vision-based measurement become more relevant in terms of prediction of the quality of food product. The conventional food quality evaluation is generally offline and destructive in nature, while image analysis is a non-destructive, eco-friendly and a non-contact technique with adequate precision.
Conclusions
The proposed work analyses the different colour features such as the red, green and blue layers of the RGB image; hue, saturation and value components of the HSV colour space; luminance and two Chrominance layers of the YCbCr colour space and the greyscale image histograms; and texture analysis feature such as the entropy-filtered image for developing a continuous freshness estimation model for oyster mushroom samples. The ANN has been used to develop a regression model for predicting the Hedonic number of a sample mushroom image. The proposed model is simple in design, and its accuracy exceeds 90% with some features, thus widening the possibility of developing a mobile application or software-based freshness estimator.