Automation is a key concept in the industry, saving working time and increasing the task’s precision and overall repeatability. It is of great importance, especially in producing items in large quantities.
Savings in working time are related to the efficiency of the plant, which can produce more in the same unit of time. The precision factor affects the production time but also often increases the prestige due to the better quality of the final product. At the same time, repeatability is a challenge that ensures consistent production of the same products while maintaining a balance in working time and precision.
These three aspects together are important from the point of view of the furniture industry. The availability of a wide range of materials and complex production systems challenge the sustainability of production.
A significant element that covers this industry is the drilling process. The materials used and the variability of parameters during work—including the structural diversity of wood and wood-based materials, mean that the level of precision may vary depending on many factors.
Here, too, the continuity of production becomes important without sacrificing quality and increasing costs.
During the drilling process, due to various factors, such as mechanical, chemical or thermal processes occurring, the drill is blunting steadily. Those are important issues in machining science. It is especially the case in materials such as melamine chipboard, where factors such as glue, the friction of wood or hard element contamination can influence this process.
Overall, tool quality, or in this case, drill bluntness, can greatly influence the quality of the final product.
Replacing the drill bit is a process that allows for reducing the damage to the material caused by the worn surface of the tool. Delaying replacement can result in poor production quality, including potential material costs. Replacing too early can also increase production costs by overinvesting in drill bits that do not necessarily need to be replaced.
The time required to exchange the drills is also a factor in that aspect, as during this process, the production is halted.
Currently, the entire tool exchange process is based on carefully calculating blade lifespan and the direct decision of the operator. Determining the optimal replacement time is also very costly, as current systems rely on estimated drill wear versus time and materials being drilled.
For an automatic production system, manual verification of drill wear is unacceptable. However, current prediction systems are inaccurate. It is, therefore, necessary to prepare a system that would allow for the correct classification of drill wear.
Various artificial intelligence solutions are widely used in the wood industry in general. Monitoring operational performance, optimising drilling parameters and AI algorithms perform well for various complex problems.
The applications of such algorithms are also growing increasingly common in the wood industry in general, especially when it comes to problems encountered during the drilling process.
A similar analysis was performed, among others, incorporating an observational approach and the application of artificial intelligence together with the use of a Convolutional Neural Network (CNN).
The implementation process required stopping the test system each time and taking pictures of the input and output holes. This was then used to determine the condition of the drill based on the effect of tearing the hole in the material.
The method based on the graphic material (photo) ensured the effectiveness of determining the drill’s wear at nearly 80 percent. However, from the point of view of automation, the process requires an additional recorder that will carry out the data collection process and transfer it to a learned system recognising the state, which will help make the right decision.
This article covers the process of developing the methodology adopted above based on the physical parameters of the drill system. The work focuses on verifying if it is possible to implement an algorithm that allows the optimisation of the drill replacement process with automatic assessment using artificial intelligence.
Materials & Methods
The measuring station is a platform with mounted standard chipboard with dimensions of 300 × 150 (mm). Inside the board was drilled a 6 (mm) deep hole.
The process uses a single-blade Faba head with a diameter of 40 (mm) with a replaceable fine-grained carbide blade. The spindle speed was 18,000 rpm with a feed per blade of 0.15 (mm).
While using the blade, there are three main states: hacking, stable state and error. Hacking is a short state directly after starting the tooling; the stable state will slowly decrease the overall tool state, while an error would result in a sudden decrease in product quality, rendering it unacceptable.
There are three different tool condition states determining the tool’s life—green, yellow and red. The Green state means new or unmarked tool (no initial data). The Red state denotes tool that requires replacement due to exceeded parameters ensuring effective operation. The intermediate state is yellow.
For precise condition determination, the drill wear intervals were adopted as VBmax. The Green state was defined as the wear level between 0 and 0.15 (mm). The Yellow state is tool wear for the range between 0.151 and 0.299. The Red state is a range greater than 0.3.
During each of the experiments, tasks were temporarily interrupted, and the current condition of the blade was subjected to physical measurements using a Mitutoyo TM-505 microscope. It is well-suited for measuring dimensions and angles.
Moreover, a Mitutoyo measuring microscope can be used to check the shape of screws and gears by attaching an optional reticule. Using this equipment, wear states have been measured and could be assigned to one of three wear states according to the following set of rules:
(1)if VBmax is in the range (0–0.15) mm, then it is a Green state—four different levels of wear state
(2)if VBmax is in the range (0.151–0.299) mm, then it is a Yellow state—two different levels of wear state
(3)if VBmax is in the range (>0.299) mm, then it is a Red state—two different levels of wear state
The experimental system has multiple sensors with the possibility of collecting 11 parameters, such as: force value in the (1) X and (2) Y-axes (3) acoustic emission (4) noise level (5) vibration level (6) device-rated current (7) device-rated voltage (8) head-rated current (9) head-rated voltage (10) servo-rated current (11) servo-rated voltage
National Instruments PCI-6111 measurement cards (for measuring acoustic emissions) and PCI-6034E (for measuring other parameters) were used for data acquisition of measurements from the sensors.
All the collected research results were divided into three data sets: DataHigh, DataLow and DataCurrent. Each set included the three mentioned above states (25 files each) and a total number of data equal 225.
The set of DataHigh included one parameter (Acoustic emission), each file with 27,999,960 records. The set DataLow consisted of four parameters (X/Y force value, noise level and vibration level), each file containing 700,000 records. The DataCurrent set included six parameters (current and voltage values for the device, head and motor drive), and the number of records was 30,000 per file.
For a better understanding of the differences between signals, spectrograms were prepared for selected input parameters obtained from the used sensors. Extreme values of the drill state are shown—Green and Red. The shown signals concern the Acoustic Emission, applied Force on the X-axis, Vibration level and Current consumption of the device.
In contrast to the optical methods of system analysis, the research carried out here allowed a collection of samples of the physical parameters of the machine. These data were imported and then processed for further use in AI methods.
The first step was to perform a 32-segment Short-Time Fourier Transform (STFT) operation to split the samples by their frequency for all 11 input variables based on their sampling frequency. To minimise data duplication, the transform did not include overlapping windows (the no overlap parameter was omitted).
A Hamming window was used to define the range. Due to the symmetry of the system, only half of the bins (segments) were used for calculations, i.e., (32/2) + 1, which gives 17 bins in total.
For each of the obtained subsets of data, the mean value, maximum and effective value (RMS) was calculated. In the end, 51 variables (3×17) were obtained for each subset signal. Thus, 11(signals)×51 = 561 variables were obtained in the entire set.
Ten Popular Classifiers
For further effectiveness analysis, classification algorithms were prepared. The goal here was to evaluate a set with available previously prepared variables for which status it will qualify. The results were verified using ten popular classifiers: K-NN (K-Nearest Neighbors), GaussianNB (Gaussian Naive Bayes), MultinomialNB (Multinomial Naive Bayes), SGD (Stochastic Gradient Descent), DT (Decision Tree), RF (Random Forest), GB (Gradient Boosting), XGBoost (Extreme Gradient Boosting), LGBM (Light Gradient Boosting) and SVC (Support Vector Machine).
The K-NN classifier is one of the most important non-parametric classification methods. In this method, the object being classified is assigned to the class to which most of its neighbours belong. In the case of an identical number of neighbours, the distances to each of them are calculated, and the smaller ‘distance’ declares belonging. The standard algorithm (based on Euclidean distance) k-NN is currently not often used. One of the approaches that improve the accuracy of the nearest neighbours classification is Neighbourhood Components Analysis (NCA). The NCA algorithm maximises a stochastic variant of the leave-one-out k-nearest neighbours scores on the training set.
The Naive Bayesian Classifier is based on Bayes’ theorem with the ‘naive’ assumption of conditional independence between every pair of features given the value of the class variable.
The Multinomial Naive Bayesian Classifier is based on Bayes’ theorem also but multinomially distributed data.
Stochastic Gradient Descent is an iterative method used to optimise the solution and its classification. It is based on the Robbins–Monro algorithm. The goal is a stochastic approximation of the optimisation of a given set (total gradient) by estimating it (randomly from a given subset). This solution is very computationally efficient for multidimensional problems but not very accurate in the convergence criterion.
Decision Tree is the simplest and most popular classifier based on scenarios of decision criteria. Narrowing down the results by range classes is the basis for decision-making. The algorithm’s performance for smaller training sets may lead to erroneous results. Decision Tree recursively partitions the feature space in the way that samples with the same labels are grouped together.
Random Forest is an example of an algorithm that uses ensemble methods. The idea of ensemble methods is to combine the predictions of several base classifiers built with a given learning algorithm in order to improve the robustness in comparison to a single estimator.
In Random Forest, each tree in the ensemble is built from a sample drawn with replacement (subset random samples) from the training set. During the splitting of each node, the best split is found from a random subset of features. Individual Decision Tree classifier has high variance and tends to overfit. Thanks to the randomness approach, the variance of the Random Forest classifier decreases.
Moreover, the injected randomness decoupled prediction errors, and by the average approach of those predictions, some errors can cancel out. The variance reduction often tends to an overall better model.
The next algorithm is Gradient Boosting, a method that uses dependencies in the previous steps of the result prediction. After each iteration, the result of the predictor is corrected for the residuals from the training set, and a new predictor is created, devoid of the error of the previous iteration.
The algorithm is the starting point for many other much-improved methods. Gradient Boosting, Gradient Tree Boosting or Gradient Boosted Decision Trees (GBDT) is a generalisation of boosting to arbitrary differentiable loss functions.
Gradient Boosting for classification is based on a regression approach, but the output cannot be a class since the trees predict continuous values, so the appropriate mapping should be applied.
Extreme Gradient Boosting (XGBoost) is an improved version of the classic solution based on Gradient Boosting. XGBoost has many advantages in comparison to standard Gradient Boosting which includes regularisation rules, parallel processing, an in-built feature to handle missing values, built-in cross-validation technique and tree pruning feature.
Another algorithm that uses Gradient Boosting is LGBM. Unlike algorithms based on random trees, such as XGBoost, it does not rely on sorting to find the best split point. It is based on Decision Trees using the decision histogram, which provides the possibility to follow the path of the expected least loss in time.
In comparison to XGBoost, LGBM has vertical growth (leaf-wise) that results in more loss reduction, and it tends to a higher accuracy, while XGBoost has horizontal growth (level-wise).
The Support Vector Machine is the learning method for classification based on correctly mapping data to multidimensional space and applying a function separating these data, declaring decision classes.
A Support Vector Machine builds a hyperplane or set of hyperplanes in a high-dimensional space based on kernel functions. The goal is to maximise the separation margin. The separation margin is the largest distance to the nearest training data points of any class (support vectors).
The entire implementation was prepared in Python programming language (version 3.9.9) with PyCharm editor enabled (version 2022.2.3 Professional Edition).
PyCharm is an integrated development environment (IDE) widely used for development. It provides functionalities such as: code analysis, graphical debugger, integrated unit tester, integration with version control systems (such as Git), etc. PyCharm is developed by the Czech company JetBrains. Additionally, scikit-learn—one of the widely used libraries, was used (open-source data analytics library).
It is the gold standard for machine learning (ML) in the Python ecosystem. This library has been applied for data preprocessing, pipeline, model selection, classifiers implementation, hyperparameters optimisation, building classification reports, confusion matrices, etc.
Pycharm has been installed on a Windows 10 system and managed the whole Python project, but python code has been executed remotely (via ssh) on an Ubuntu 18.04.6 LTS (Bionic Beaver) machine that is dedicated to machine learning and deep learning projects. All experiments were performed on hardware (Ubuntu operating system).
Summaries
In the conducted research, the ten previously mentioned classification algorithms were used in the cross-validation method on the input sets of the collected data. Each set of data (DataHigh, DataLow, DataCurrent) for each state (‘Green’, ‘Yellow’ and ‘Red’) was prepared using a Pareto rule (also called 80/20), which was implemented learning on 80 percent of the data set and testing was performed on the remaining 20 percent.
The spectrograms of the selected time courses show clear changes in the tendency of signals, which, however, can be correctly interpreted. A face-to-face comparison of the extreme states shows how the signal changes with each drilling job for a drill marked as new or excellent (‘Green’) and one that is worn (‘Red’).
In the case of Acoustic Emission (DataHigh) for state ‘Green’ (a), slight changes in the value of the frequency level can be noticed in the final stage of the task. For a drill marked with state ‘Red’ (b), the emission level in the analysed frequency range is distributed over the entire time interval of the task stage, which may mean that the source of interference is in the indicated drill.
For the spectrogram of the applied Force in the X-axis (DataLow), both in the case of a drill with the state ‘Green’ (a) and ‘Red’ (b), it is uniform throughout the task. However, there is a visible change in the force value, which increased six times in the case of a worn drill bit.
The system’s vibration level (DataLow) corresponds to the mentioned acoustic emission—for the ‘Green’ drill (a), the vibration level is not only low but also slightly increases at the end of the stage works. For a worn drill bit (b), the vibration level is much higher (up to four times) and is present throughout the task.
The last spectrogram comparison set of all 11 features available is the current consumption value (DataCurrent) of the device during the task.
According to the principle of operation of the electrical device, the current consumption increases only for the system that starts operation, and in the case of achieving full stabilisation, the value drops at the very end of the task. This can be seen from the spectrogram for state ‘Green’ (a). In the case of a worn (b) drill, the value of the current consumption varies and can be read as unstable operation of the device.
As a result of the analysis, summaries were developed for each classification method, including the precision of classifying the state to its actual class in relation to the assumptions. Each misclassification against the expected value resulted in a lower prediction level for a single state (e.g., expected ‘Red’ has been classified as ‘Green’) and the entire set (e.g., how many times ‘Red’ is classified as ‘non-Red’).
The first of the considered parameters for the solution quality is the overall algorithm accuracy. Out of the total results, XGBoost (Extreme Gradient Boosting) is the classifier with the highest prediction score, 93.33 percent. Two more classifiers: GB (Gradient Boosting) and DT (Decision Tree), achieved a result of 86.66 percent. The remaining methods did not exceed the threshold of 85 percent, which can be considered a poor result.
Algorithm XGBoost chose only 50 features to build the model. The ranking features are depicted. The number at the end of the feature name is regarding the number of the segment in the STFT algorithm (maximum 17). The main signals that XGBoost used to build the model are Force X, Noise, Ac. Emission and Vibration.
Apart from the overall accuracy, the misclassification rate is another important factor in defining the quality of the final results.
Analysing the Confusion Matrices, it can be clearly stated that in the case of the XGBoost classifier, there were FP cases (False Positive) for state ‘Yellow’, which defined the wear of the drill as good (‘Green’). However, it was marked as already worn. Taking into account the degree of precision, the error is significant, but it still does not declassify the algorithm as suitable for the adopted methodology.
This level of accuracy allows the method to be put into practice with certainty. In the case of algorithms that obtained second and third place in terms of classification accuracy (achieving the same score), there were cases of incorrect cross-classification. The ‘Yellow’ state was assigned to the ‘Red’ drills and vice versa. Such scenarios can result in real losses in the case of this type of error.
What is most important is the lack of ‘Green-Red’ and ‘Red-Green’ errors in the best-performing algorithms. From the tool wear and overall production process point of view, such misclassifications have the most impact on the solution quality.
In the presented cases, no such errors occurred during method evaluation. Additionally, in the case of the XGBoost algorithm, both border states (‘Green’ and ‘Red’) had 100 percent accuracy when being assigned, with the only errors occurring in the ‘Yellow’ state.
This minimises the two main risks: first, the tool being exchanged too early (or too often), increasing the production downtime; secondly, the tool being in unsatisfactory condition to be continuously integrated in the production process, resulting in poor quality products and loss for the manufacturer.
Overall, the misclassification rate, when combined with high accuracy obtained by the best methods, provided a viable solution that is applicable to the presented task.
Advantages & Improvement
The article presents a new approach to the problem of drill wear classification. The tests were not based on a visual representation of the holes but on measurements of the drilling system in terms of physical parameters, such as noise levels, current/voltage values and vibrations.
Each of the measurements was saved in a separate data set, the values of which were parameterised in the time window corresponding to the changes in the real research interval.
The presented algorithms determined the level of accuracy in this particular study. Of the available classification solutions, three achieved a precision score above 85 percent. This value is satisfactory from the industry’s point of view, but it may be too low to minimise the damage and costs of drilling holes in wood and wood-based materials in this case.
The XGBoost classification algorithm was the only one to achieve a precision value above 93 percent on all 1 time windows and test sets. Such a result indicates that this methodology can determine the practical implementation in production.
The XGBoost algorithm was also very accurate in terms of misclassification errors, as the only state that was not correctly assigned was the ‘Yellow’ one. The lack of ‘Red-Green’ and ‘Green-Red’ errors is an additional advantage of the presented method.
The proposed solution has a very high level of certainty, but it is still possible to improve and re-verify in the case of an analysis based on real, more extensive sets of data. Such information may come from furniture factories or precision wood processing plants, increasing the diversity and overall representation of the processing and environmental properties.
