The granite manufacturing industry holds a key role in the industrial network of the North-Western area of Spain. This region is the largest ornamental granite producer, accounting for 62% of the total national production (1). Traditionally the mining sector, including the granite industry, has been reluctant to adopt new technologies, but the competitive market leaves few options for the mining industry (2). In this context, an improvement in traceability is required. By definition, traceability involves any processes, procedures, or systems that support the generation of verifiable evidence about a product as it moves along its supply chain (3). Nowadays, blocks are initially identified in the quarry with marks indicating type and origin, once they reach the factory for further processing, they are colour-coded for identification purposes (4). Given this setting, a computer vision application could prove helpful in the automation of the traceability system in use.

The scope of computer vision is the development of theories and algorithms for automating the process of visual perception. The mathematical basis for image processing and feature analysis was defined by several authors, including Ervin E. Underwood, John C. Russ, and Jean Serra (5-8), in the last decades of the twentieth century and beginning of the twenty-first, while its applications reach a broad range of fields, and the potential benefit of its development can have a major impact in the upcoming years. Computer vision has found its applications in the mining sector, more specifically, in the slate industry for the detection of defects in the ceramic industry (9-11), for the same purpose in the marble industry (12-16), for pattern detection and classification (17, 18) as well as in the granite manufacturing industry for the characterization of granite varieties (4, 19), but there has not been found any previous research on the applications of computer vision for colour detection and analysis in order to improve product traceability in this sector.

The current traceability method based on colour-coding each granite block is affordable but entails a substantial number of drawbacks, which may lead to failures. The colours have to identified and interpreted as a sequence of integers several times in the production process by an employee. This simple task can be subject to human error due to fatigue and the harsh work environment in the granite industry, plus its accuracy is affected by the weathering of the colours. The main goal of the program developed is to analyse pictures of granite slabs with the colour code drawn on their side and convert the colour bands into their corresponding numerical code. Consequently, the implementation of a program of this kind would dramatically reduce the time needed to decrypt the colour code, currently performed manually, and reduce human error. In addition, it decreases economic losses by avoiding the discard of those slates not correctly identified. The work presented in this research paper is a crucial step in the implementation of this system on mobile phone devices for in-factory use with the same purpose and even more features.

Section 2 of this paper introduces the materials needed for this research, including the granite slabs, the images with the colour bands, a middle-class computer, and Python 3.9. and explains the methodology of this research, focused on the algorithm developed. Section 3 presents the results obtained on the detection of colour bands on granite and the decryption process. Lastly, Section 4 concludes with the main findings and future applications.

Granite is by definition a very hard natural igneous rock formation of visibly crystalline texture formed essentially of quartz and orthoclase or microcline and used especially for buildings and monuments. It typically contains 20-60% quartz, 10-65% feldspar, and 5-15% mica (biotite or muscovite), although a wide range of lithological materials is considered granite from a commercial perspective (4). Granite blocks mined in the quarry are wired cut when processed in the factory to produce the granite slabs. Their dimensions are not consistent and tend to range between 1.70 to 2.00 meters in width and 20 to 30 mm in thickness.

The database used in this research project consists of 130 pictures of granite slabs focused on one of the sides of the slab, where the colour code displayed is in the shape of bands manually drawn. The sole purpose of this particular type of code is to keep track of the granite slab from its entry into the factory until its final sale to the customer.

The colour code is made up of a variable number of colour bands drawn using spray paint on the granite block once it enters the factory. They are placed either close to the higher or lower edges of the block, with a distance of approximately 15 cm in between them. Each band can have any of the 8 different colours shown in Table 1, and they can be placed in any order. Each colour is associated with a number, which makes the block have a specific numerical code depending on the configuration of the colour bands it has drawn on its side. This method to numerate, count and keep track of every block has been chosen over simply drawing the corresponding number on the block because, to produce the final version of the product, the block has to be wired cut. This means that the colour bands drawn on the surface of the stone would be cut as well, and they would become illegible, losing their purpose.

The iPhone 12 Pro camera used to capture the side of the granite slabs in the RGB colour space has a resolution of 12MP, a ƒ/1.6 aperture, and 26 mm of focal length. Development and testing of the program have been completely performed with a 1.6 GHz Intel i5 10210U processor with 16 GB (2×8 GB) of RAM. The program to identify colour codes is written in Python 3.9 using OpenCV, which is an open-source computer vision library written in C++ and C.

The training process for each function defined the ideal values for the parameters that the colour detection functions depend on. These results are presented in Table 3, where the first column contains the name of the different colours present on the slabs. The next four columns contain information regarding the geometrical characteristics of the colour bands, and the rest of the columns present the hue, saturation and value limits for each colour threshold.

It is noticeable how, in the training process, the colour ratio (CR) between the number of active pixels after applying the thresholds and the total number of pixels in the image tends to zero. This is due to the presence of some faint and weathered colour bands that would not be detected otherwise. If CR took higher values, some of these images, which actually contain areas of interest, would get discarded and count as a classification error. Additionally, the results presented in Table 3 show how the hue measure is the most effective in isolating different colours compared to the saturation and value. The difference between the maximum and minimum hue values is the smallest in all colours except black, which need to be isolated with the aid of the value variable. This reduces the importance of the two other parameters (saturation and value), which leads to them having more wide ranges explaining the reasoning behind the higher limit of the saturation and value threshold having the maximum number possible (255) in almost all cases. An exception to this rule is the colour black, which has to be isolated with the aid of the value parameter.

The conditions and the success rate of every colour in the training phase along with the computation time are included in Table 4. During the development of the program, it was noticed that the colours red and brown display two different varieties. The changes in the shade of brown have their origin in the weathering of the colour, resulting in two categories: dark brown and light brown. In the case of the red colour, we hypothesize that its variance is heavily dependent on the brightness conditions under which the picture was taken. The presence of clouds in the sky or shadows projected on the granite slabs seems to be a likely cause for a high hue value, while if the light rays hit the granite slab and the shadow is projected backward the value of hue for the red colour tends to be much lower as it is shown in Table 3. In order to improve the accuracy, a double mask method was implemented in the algorithm, which works by defining two distinct thresholds for each colour variety but still identifying them as the same colour.

The results of the validation process for each colour function are presented in Table 5. It can be seen in the fourth column that the times required for each calculation decrease in direct proportion to the number of images analysed. The images in this set are completely new for the algorithm, meaning that it has not been trained on them. Therefore, the high success rates in this process confirm that the parameters defined in the training stage are accurate, and the model can read the correct codes of new images which makes it valid.

After the training and validation of all colour detection functions, these are included in the main architecture of the final algorithm, which is tasked with the detection of each colour band and its conversion into the correctly ordered numerical code. In this case, a result is classified as true when the output of the algorithm matches the exact numerical code an employee would read off the granite slab. Those codes obtained from the algorithm whose numbers are not the same and/or are not placed in the right orders would be considered faults. The colours which present a lower success rate tend to have an important negative effect on the final result. Particularly, light brown can be mistaken for orange, and dark brown can be confused with the low-hue red due to their proximity on the HSV colour space. These conditions make it difficult for the algorithm to achieve a very precise detection on all colours, which hijacks the results leading to a faulty detection, as can be seen in Figure 6.

The replacement of the colour brown is proposed in order to tackle this issue. The spray paint colours cyan and light green are encouraged due to their strong contrast with the current colours in use, as shown in the HSV hexacone contained in Figure 1. Moreover, the election of the colour pink was motivated by its great endurance to weathering, as it is demonstrated by Alonso-Villar et al. (36). According to these authors (36), the use of silicate-based paint can be considered as a future line of research to solve the detection problem associated with the erosion of the spray paints currently employed.

Additionally, most colour detection systems have to operate within steady light conditions (3, 5, 37, 38) and require stable camera positioning (39-42), while the system proposed, although not perfect, achieves results in very different conditions. In this case, the algorithm shows a success rate of 74.42% on the training set, while the validation shows an efficiency of 75.00%. Several examples of correct detection are displayed in Figure 7.

Several results of the validation process for the final algorithm are shown in Table 6. The first column contains the ground truth for each image, which is the numerical code defined by the colour bands in each case. The last column shows the numerical code decrypted by the algorithm from the information represented in the corresponding image of the granite slabs. The rest of the columns contain each of the sorted numbers outputted by the system. According to the success condition previously defined, a result is correct when all numbers match the ones present in the ground truth in the correct order. Two errors can be noted in red in Table 6. The result of the fourth code presented (24036) is affected by the detection of a black area out of place, and the same happens in the case of the fifth code of the table (24066). Consequently, the rest of the numbers in the code decrypted are displaced by these anomalous detections of black areas leading to a faulty result. Considering the fact that granite has important black, areas in its structure due to the presence of mica, this becomes the main source of errors, despite the final algorithm having an accuracy of 75.00% in the testing phase.

The traceability problem in the granite industry has been studied in the present research paper. The current method to keep track of the granite blocks until they reach the final product stage is fairly simple and cheap, consisting in the usage of graffiti spray to assign a color-coded number to each granite block. Weathering of the paint and human error due to different causes, such as fatigue, lead to a faulty reading of the numerical code represented by colour bands and ultimately to economic losses for the industry as well as pollution originated by the granite that has not been traced. In this paper, a computer vision algorithm is proposed to automate the traceability process in this industry. The computational method developed operates in the HSV colour space and makes use of a double threshold for the correction isolation of colours on the granite slabs. This is followed by the conversion of the thresholded image to grayscale and the application of Gaussian blurring to soften the edges. After the binarization of the blurred image, contours are detected and the coordinates of those shapes that approximate to rectangular colour bands are extracted. Lastly, those coordinates are sorted and converted into the decrypted numerical code.

The proposed system performs an accurate detection of a combination of seven out of the eight colours used in the granite industry. It shows success rates on the reading of colour codes and the output of their corresponding numerical codes of 74.42% in the training set and 75.00% in the validation set. Moreover, these results are achieved on images that display different lighting conditions and were not taken with a fixed target distance, which increases the difficulty in the detection of different colours and adds up to the value of the results presented. In conclusion, the computer vision system presented is considered to be helpful in the granite manufacturing industry owing to its accuracy, and it is part of the first approach towards a full implementation on mobile phone devices for the corresponding users in each factory.