Comparison of artificial intelligence methods for predicting compressive strength of concrete

Compressive strength of concrete is an important parameter in concrete design. Accurate prediction of compressive strength of concrete can lower costs and save time. Therefore, thecompressive strength of concrete prediction performance of artificial intelligence methods (adaptive neuro fuzzy inference system, random forest, linear regression, classification and regression tree, support vector regression, k-nearest neighbour and extreme learning machine) are compared in this study using six different multinational datasets. The performance of these methods is evaluated using the correlation coefficient, root mean square error, mean absolute error, and mean absolute percentage error criteria. Comparative results show that the adaptive neuro fuzzy inference system (ANFIS) is more successful in all datasets.


Introduction
Concrete is the most extensively used construction material in the world due to its various advantages. Large scale production of concrete contributes to depletion of natural resources (sand, gravel or crushed stone). Furthermore, the emission of greenhouse gases increases with an increase in consumption of cement [1]. Supplementary materials are therefore used in concrete production to decrease consumption of cement and natural resources. Prediction of mechanical properties of construction materials has become an important area of research in materials science and civil engineering. In practice, concrete is classified according to compressive strength. Therefore, an accurate estimation of compressive strength of concrete is important. Concrete is made up of different constituent materials (aggregate, cement, water and supplementary materials), and these materials are randomly distributed in concrete matrix. Due to this complex structure of concrete matrix, the number of effect variables affecting the concrete compressive strength is quite high and, therefore, the predictability of the concrete compressive strength is quite low [2,3]. However, with the help of developing technology, the prediction accuracy of concrete compressive strength can be increased by artificial intelligence (AI) methods. In recent years, many studies have been made in order to predict compressive strength of concrete based on its constituents [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19] and also using simultaneously controllable effect variables [20,21]. Due to sustainable production and cost, various additives (supplementary materials) are used in concrete production, which makes it difficult to achieve the desired concrete properties. Therefore, it is important to determine the usability of AI methods for a highly accurate prediction of concrete compressive strength in the design of concretes containing supplementary materials. Topçu and Sarıdemir [4] and Başyiğit et al. [6] used artificial neural networks and fuzzy logic models for predicting compressive strength of concrete containing fly ash. In addition, Sarıdemir [5] compared artificial neural networks and fuzzy logic models for prediction of compressive strength of mortars containing metakaolin at the age of 3, 7, 28, 60, and 90 days. Sarıdemir [7] developed two models in the gene expression programming (GEP) approach for the prediction of compressive strength of concrete containing rice husk ash at the age of 1, 3, 7, 14, 28, 56 and 90 days. Gilan [22] developed the support vector regression (SVR) [23] and particle swarm optimization (PSO) [24,25] model for the prediction of the compressive strength, and used rapid chloride penetration test results of concretes containing metakaolin. Atici [26] used multiple regression analysis and an artificial neural network to predict compressive strength of concrete containing various amounts of supplementary materials (blast furnace slag and fly ash) at different curing times (3,7,28,90, and 180 days).
The analysis of optimization problems of high-strength concrete is important in concrete industry, and so these problems are investigated with multi-objective optimization based on regression analysis, artificial neural network (ANN), and gene expression programming (GEP) [27]. Moreover, support vector machine (SVM), fuzzy logic algorithm and ANN are used to predict elastic moduli of normal and high strength concrete [28][29][30]. Artificial intelligence methods are also used to determine properties (compressive strength) of self-compacting concrete. Zhang et al. [10] proposed a beetle antennae search (BAS) algorithm based random forest model to significantly predict compressive strength of the lightweight self-compacting concrete. Siddique et al. [9] compared data taken from literature and developed experimentally, using neural network techniques for the prediction of compressive strength of self-compacting concrete. Moreover, Ghafoori et al. [31] compared the linear-nonlinear regressions and neural network for predicting the rapid chloride permeability of self-consolidating concretes based on proportions of their constituents. Sobhani et al. [8] presented the use of more reliable ANN and ANFIS models for the prediction of 28-days compressive strength of no-slump concrete. Tsai [32] proposed the hybrid multilayer perceptron models for predicting strength of concrete specimens. Tanyildizi and Çevik [33] used genetic programming for predicting the compressive and splitting tensile strength of lightweight concrete (cement contents: 400 and 500 kg/m 3 ) exposed to high temperatures. Feng et al. [11] proposed a new approach based on the adaptive boosting algorithm for predicting compressive strength of concrete.
A general successful prediction model, which can be applied regardless of the dataset, is presented in this study. Six different datasets with different input variables and sample sizes are used for this purpose. One of these datasets was produced by the author. ANFIS, random forest (RF), linear regression, classification and regression tree, support vector regression, k-nearest neighbour and extreme learning models, are used to predict compressive strength of concrete. The correlation coefficient (R), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and P-value criteria are used to evaluate the success of AI methods used in this study.

Prediction models
In most engineering problems, the variables that make up the problem are dependent on each other. When one of the variables changes statistically, other variables do not change to the same extent. It is very important to reveal this causal relationship between the variables. Creation of the regression GRAĐEVINAR 73 (2021) 6, 617-632 Comparison of artificial intelligence methods for predicting compressive strength of concrete equation for predicting the most accurate way of estimating the value of a random response variable depending on the value of another effect variable or variables is called regression analysis. Nowadays, the use of artificial intelligence methods is rapidly increasing due to reasons such as the size of the datasets, interaction of multiple complex variables, nonlinear relationship between the input and output (target) variables, and predictive accuracy performance of the output variable. Regression analysis is a subfield of supervised learning. In the supervised learning process, the algorithm is trained with a dataset whose inputs correspond to the outputs. The independent test dataset is used to evaluate the prediction success of the trained algorithm. The flow diagram for regression analysis is given in Figure 1. Statistical computations and development of models were carried out using the R programming language [34].

Random Forest (RF)
The random forest method is an ensemble learning method used for classification and regression. In this method, many decision trees are created during the training and then an average of the results obtained by these decision trees is used during prediction. The most important advantage of the random forest method is that it provides a solution to the problem of excessive adaptation in decision trees. This method can easily be parallelized as it is simple, and is more resistant to extreme values and noise. It is also more successful than other ensemble learning methods [35,36]. In this study, caret [37] R package was used for the implementation.

Classification and Regression Trees (CART)
CART is a decision tree algorithm. It is used for solving classification and regression problems. The purpose of the decision tree is to create a tree model that can estimate the output variables value using input variables in the dataset. The CART algorithm is a decision tree algorithm that uses the Gini coefficient as the division criterion, utilizing the binarydivision type, using categorical or continuous variables [38]. In this study, rpart [39] R package was used for the implementation.

Support Vector Regression (SVR)
In the Support Vector Regression, the aim is to find the most possible linear function that estimates a group of data in space with as much error as epsilon (a margin of tolerance). In case the data can be separated linearly, the data can be separated directly by a hyperplane which maximizes the margin. In case the data cannot be separated linearly, the data is mapped to a higher dimensional space with the kernel function. Thus, the data is provided for linear separation. As a result, it has F(x) function for converting training inputs into outputs [23]. In this study, e1071 [40] R package was used for the implementation.

K-Nearest Neighbour (K-NN)
K-NN is a sample-based algorithm that can be used for solving classification and regression problems. Distances are calculated between the test data sample and all samples in the training dataset, in order to find the nearest "k" sample. Then the output variable is calculated by averaging these nearest samples [41]. In this study, FNN [42] R package was used for the implementation.

Extreme Learning Machines (ELM)
Extreme learning machines are basically similar to ANN with one hidden layer. Therefore, the working principle of ELM is almost the same as the working principles of ANN. In the ELM training phase, weights and bias values are assigned from the input layer to the hidden layer randomly, and these values do not update in the whole process [43]. As in artificial neural networks, an activation function is also needed for the ELM. Various types of activation functions are available, and can be selected by the user [44]. In this study, elmNNRcpp [45] R package was used for the implementation.

Linear Regression (LR)
LR is used in many studies due to its ease of use, efficiency, and effectiveness. Simple linear regression equation (y = w 0 + w 1 x 1 + w 2 x 2 + ... + w n x n ) expresses a linear relationship between an independent variable (x 1 , x 2 , …, x 3 input variable) and a dependent variable (y, output variable). This model fits straight-line models between each input variable and output variable [46]. In this study, stats [47] R package was used for the implementation.

Adaptive Neuro Fuzzy Inference System (ANFIS)
The ANFIS method is based on the fuzzy inference system. ANFIS is a model that uses fuzzy logic and artificial neural networks. This method is an effective prediction model between neuralfuzzy systems and machine learning techniques [48]. ANFIS learning algorithm consists of the least-squares method and the backpropagation algorithm. Input samples are produced in the first step of the learning process, and the best secondary parameters are determined using the least squares means method. Here, the primary parameters are assumed to be constant. In the second step, the input samples are duplicated and replaced with the gradient descent method of the primary parameters, assuming that the secondary parameters are constant. This process is then repeated [49][50][51]. In this study, frbs [52] R package was used for the implementation. The ANFIS structure is given in Figure 2. In Layer 1 (fuzzification layer), the output of each node is a membership function such as Gaussian, triangle, trapezoidal, etc. In Layer 2 (rule layer), the output of each rule node is multiplied by the membership grades using product operation and the firing strength rule is applied in calculation. In Layer 3 (normalization layer), all nodes coming from the rule layer are accepted as input values and the normalized firing strength of each rule is calculated. In Layer 4 (defuzzification layer), weighted values of the rules are calculated. The overall output, being summation of all the rules, is calculated in Layer 5 (summation layer). This layer contains only one fixed node [53].

Performance measures and validation methods
Various splitting strategies can be applied to train the model and test its performance [54,55]. The validity of the learned model is different from the algorithm validity. The generalization ability of the model depends on its performance on unseen test data. In this study, the dataset was split using the 2/3 + 1/3 approach. Randomly selected 2/3 of the dataset were used in model training and the remaining 1/3 was used for the performance of learned models. Models were run 10 times and after 10 runs, the misprediction error ranges of the models were given (Table  4). Then, in order to compare the prediction performance of all models, the datasets were randomly separated from the same point with the seed approach. R, RMSE, MAE, and MAPE performance measures were used for validating performance of the learned AI models on unseen test sets ( Table 5). The analyses were conducted in R programming [34]. Correlation coefficient (R): The R measure shows the degree of linear correlation between observed (actual/measured) and predicted compressive strength values. The R-value ranges from 0 to 1. As the prediction accuracy of the model increases, the R-value approaches 1. It is expressed as follows: where O is the observed value of compressive strength, is the mean of the observed value, P is the predicted compressive strength value of the developed artificial intelligence model and is the mean of the predicted value. Root mean square error (RMSE): Indicates the standard deviation of the difference between the observed and predicted compressive strength values. Smaller RMSE value is desirable. It is expressed as follows:

Data description
The prediction accuracy of the models was evaluated using published multinational datasets [22,[56][57][58]. For all datasets, compressive strength is the output variable, and input variables vary according to datasets. The statistics of the datasets is given in Table 1.   [20,21,60]. Input variables in dataset 1 were selected as quantities of constituent materials of the concrete samples produced within the scope of the PhD thesis [60]. These concrete constituent materials have not so far been published or used in any research. In normal concrete mixes, three different types of cement were used as binding materials, and superplasticizer was used to ensure sufficient workability. Properties of binding materials and superplasticizer used in dataset 1 are given in Table 2. Variables of dataset 1 are given in Table 3. An average of three sample results was used for each specific mix design point in dataset 1. The samples were cured in lime-saturated water at 23±2 °C for 28 days and then tested. Dataset 2 was produced to determine the predictability of f c and rapid chloride penetration values of concretes containing metakaolin by using the hybrid support vector regression (SVR) -particle swarm optimization (PSO) model [22]. Moreover, the hybrid model was compared with an adaptive neural-fuzzy inference system (ANFIS) [22]. ASTM C150 Type-I Portland cement, and metakaolin obtained from three kinds of kaolin, were used in all concrete mixtures [22].
Dataset 3 was produced for the investigation of strength, compressive stress-strain relationship and fracture behaviour of concrete by replacing cement with fly ash and silica fume in certain proportions [46]. ASTM Type I Portland cement, ASTM Class F low calcium fly ash and silica fume were used in all concrete mixtures [56]. Moreover, a naphthalene-based high range water-reducing admixture (HRWRA) was used in mixes to ensure adequate workability [56]. No artificial intelligence method was used to analyse dataset 3 by Lam et al. [56]. Dataset 4 was produced for predicting f c of high-strength concrete (HSC) by using the extreme learning method [57]. Moreover, the results of the extreme learning method were compared to the results of artificial neural networks (ANN) [57]. Type 1 Portland cement and polycarboxylate superplasticizer were used in mixes [57]. Dataset 5 was produced for reducing the number of trial mixtures of the high-performance concrete mixtures (concretes with a compressive strength variation range of 40-80 MPa) by using a genetic algorithm [58]. ASTM Type I Portland cement, class F fly ash, and naphthalene superplasticizer, were used in mixes [58]. Dataset 6 was produced for predicting f c of high-performance concrete by using artificial neural networks (ANN) [59]. ASTM Type I Portland cement, fly ash, blast furnace slag powder, and superplasticizer containing naphthalene-formaldehyde and fatty acid copolymer, were used in mixes [59].

Results and discussion
The correlation coefficient gives information on the effect level and direction of the linear relationship between two parameters. The cross-correlation matrix of the six datasets is shown in Figure 3. According to the correlation results of dataset 1, the output variable f c is highly correlated with the effect variable f cc (0.81). This result is similar to results obtained in previous studies [20,21,60]. In normal concretes, the effect of cement content on the concrete compressive strength is expected to be high. In addition, the water/cement ratio is one of the most important parameters affecting f c . The variation interval of cement content is very low (345-330 = 15 kg), while the variation of the water/ cement ratio is mainly due to the variation in the quantity of water. Therefore, while the correlation coefficient of water content (W) is in the order of -0.26, the correlation coefficient of cement content (C) is equal to approximately zero (-0.01). The coarse aggregate is positively correlated (0.42) with f c since it forms the bearing frame of normal concrete. As fine aggregate forms the mortar phase in normal concrete, there is a negative correlation (-0.39) between the fine aggregate and f c . The correlation coefficient of fine aggregate is 0.01 when the simultaneously controllable variables are selected as input variables [20]. It is therefore quite difficult to generalize the correlation coefficient because the effect levels of the input variables vary according to the selected input variables and variation intervals. The lowest correlation coefficient belongs to the SF input variable. The increase in the amount of SF, which exhibits bindingness due to its very fine structure and high amorphous silica content, caused only 0.10 increase in f c due to the fact that the SF variation interval has a range of 0% -5%. Furthermore, since A has a 3-180 days variation interval (correlation coefficient of A is 0.68), the correlation coefficients of mineral additive materials and total binder material with the compressive strength do not reflect the actual effect levels. Therefore, classification should be preferred when determining the change interval of A, especially in concretes with mineral admixture. Due to the water/binder ratio in concretes with mineral admixtures, the variability in water content significantly affects the compressive strength (correlation coefficient of W is -0.57). In addition, the f c and HRWRA exhibit a positive correlation of about 0.49 to achieve workability due to the increased fineness of total binder materials. FA and W variables have a negative correlation (-0.77). This does not reflect the physical interaction between the amount of fine material and water requirement. However, when the positive correlation between FA and HRWRA variable (0.62) is taken into consideration, it can be seen that the loss of workability due to fineness is achieved  In the study, after running the models 10 times, the training and test error ranges are given in Table 4. In addition, datasets were separated from the same point in order to compare predictive performance of all models. Accuracy performances of all models according to R, RMSE (MPa), MAE (MPa), MAPE (%) and P-value criteria on unseen data are presented in Table 5, Figure 4, and    Figure 4. Due to high prediction accuracy of the ANFIS model, high R 2 values were obtained for all datasets (R 2 range from 0.9938-0.9631). R 2 shows the degree of linear correlation between the observed and predicted f c value. It can be stated that the closer this value is to 1, the model's prediction ability is the higher.
On the graphics given in Figure 4, points distributions are close to the line, which indicates that there is a strong correlation between the observed and predicted values of f c . Taylor graphics [61] obtained for each dataset are presented in Figure 5. Prediction performances of all AI models are compared in these graphs. In the Taylor diagram, correlation coefficients, standard deviation, and RMSE metrics, are used to determine comparability between AI models. As shown in Figure 5, ANFIS is the superb prediction model in which the data closest to the observed values with low RMSE and high correlation coefficient are obtained.

Conclusions
The predictability of concrete compressive strength is very low due to the use of many different materials in concrete production, and the complexity of concrete structure. The importance of the high prediction accuracy of f c is well recognized in the ready mixed concrete industry due to the complex structure of concrete. Thus, researchers have proposed many models for improving the prediction accuracy of f c [20, 22, 57-59, 62, 63]. In this study, different AI model prediction performances were compared for six different datasets in order to determine a general successful method in concrete compressive strength prediction independent of the datasets. R, RMSE, MAE and MAPE metrics were used for model performance comparisons. The relationship between input variables in the datasets and concrete compressive strength was also examined. As a result of the study, the following results were obtained: Correlation coefficients are directly affected by variation intervals of input variables. Therefore, in order to determine actual levels effects input variables have on output variables, variation intervals of input variables should be selected carefully. Especially, the contribution (effect) of component materials to f c in high strength and mineral added concretes may not be accurately determined due to the variation interval of the ages input variable. Classification of dominant input variables (ages, etc.) in the experimental designs will be useful to determine actual output functions. High prediction accuracy is obtained by using artificial intelligence methods for different concrete types. The highest prediction accuracy was obtained with the ANFIS method according to all evaluation metrics for all datasets (average R = 0.992, RMSE = 1.81, MAE = 1.409, MAPE = 4.06%, P<0.001). RF model results are close to the ANFIS and the lowest estimation accuracy was obtained by the CART model. This success of the RF model shows that the ensemble learning method is more successful than a single decision tree. The correlation coefficients and prediction accuracy cannot be generalized for the effect levels of the input variables on the output variable, and they vary completely according to the selected input variables and the variation intervals. The prediction performance of machine learning methods depends on many situations. The performance of the model can be affected by many factors such as data pre-processing, model hyperparameters, and splitting strategies of the dataset. The significance of the method (p<0.0001) and evaluation metrics are not sufficient to physically indicate that the model is significant. Therefore, the developed models are valid only for the selected input variables and the variation interval of the input variables as well as data used for training/testing. In the context of total quality control, the usability of artificial intelligence learning methods should be improved in the readymixed concrete industry, since high prediction accuracy will contribute to the reduction of losses that may occur in the production of ready mixed concrete.
In practice, concrete component material properties change depending on seasonal parameters and material sources. Therefore, the prediction accuracy of concrete compressive strength should be determined in future research by using datasets not only produced under laboratory conditions but also including the variations in the component material properties. In addition, it is necessary to make a classification by taking into account the functional (workable) regions of the simultaneously controllable effect variables, and the prediction accuracy of the models should be determined separately for each region.