Effects of Combination of Novel Feature Engineering Technique with Random Forest Feature Selection and Variance Inflation Factor Reduction on Prediction Accuracy
Feature engineering plays a crucial role in improving machine learning model performance. This study presents a novel feature engineering technique using Fibonacci linear and polynomial equations of degree 7 to enhance predictive accuracy, the linear equations are employed to retain the structure of the original glass dataset which determines which of the series of polynomial equations will be used to transform the dataset. The aim is to compare the results of this technique with further combining it with statistical method. The engineered features were combined with the original features, random forest feature selection was used to select features from the combined feature and was evaluated, this achieved 81% accuracy. The polynomial equation used are variant polynomial regression which are susceptible to multicollinearity, the random forest selected features was further subjected to reduction using Variance Inflation Factor (VIF) to eliminate multicollinearity while maintaining interpretability, this achieved an accuracy of 86%. The final model achieved significant accuracy improvements while ensuring reduced redundancy among features. The findings highlight the comparative effectiveness of combining Fibonacci-derived polynomial feature expansion, Random Forest selection; and Fibonacci-derived polynomial feature expansion, Random Forest selection with VIF reduction in predictive modelling.
Download Paper