Osteoporosis is a disease characterized by decreased bone density resulting from a higher rate of bone loss compared to bone formation, which increases the risk of fractures. Therefore, early detection of osteoporosis helps reduce the risk of fractures. Traditional methods for diagnosing knee osteoporosis are time-consuming and require specialized personnel. Machine learning has emerged as an effective tool for image analysis, we proposed in research the development of an intelligent system for detecting knee osteoporosis using X-ray images. The research included both binary classification and multi-classification to differentiate images into five main categories: normal, Doubtful, mild, moderate, and severe osteoporosis, using two separate datasets. We proposed a new feature extraction technique called Hybrid Multi-Modal Feature Fusion (HMMFF), which integrates traditional feature extraction techniques, such as GLCM, HOG, and ORB, with deep learning models, including YOLO11, VGG-16, and ResNet50. Feature selection was performed using the harmony search optimization algorithm, and the selected features were then fed into several machine learning classifiers for training, evaluation, and classification. Experiments showed that the XGBoost model achieved the best performance compared to the other models, with an accuracy rate of 94.5% in binary classification and 93.3% in multi-class classification. The results indicate that the application of the HMMFF technique contributed to the extraction of comprehensive and diverse features. Furthermore, the feature selection using harmony search optimization helped identify important and effective features, thereby increasing the accuracy of classification and diagnosis of osteoporosis and consequently reducing the risk of fractures.