Multimedia and Virtual reality technology Division

Address: Room 610, House E3, 144 Xuan Thuy, Cau Giay, Hanoi

Phone: (024) 37547347

PGS.TS. Lê Hoàng Sơn
Manager

The Department of Multimedia and Virtual Reality (Department of Multimedia and Virtual Reality) undertakes the main tasks of research, technology development and training of core modules in the training program. specializes in Computer Science and Management Information Systems at doctoral and postdoctoral levels. With the mission of developing top-notch research and training high-quality human resources capable of participating in the process of creating new knowledge and products for the 4.0 technology revolution and digital transformation of the country. , especially related to the main research direction of Artificial Intelligence (AI), Soft Computing, Image and Video Mining, Virtual Reality and Augmented Reality (Virtual). Reality and Augmented Reality).

Through the process of research and selection, the strong research directions of the Multimedia and Virtual Reality Technology Research Department include: Artificial Intelligence (AI), Soft Computing, Processing. Image and Video Mining, Virtual Reality and Augmented Reality, Algorithms and Optimization, Evolutionary Computing, Performance Computing High performance in image and video processing, Fuzzy logic and computation, Distributed Computing, Machine Learning, and specialized data analysis (Business) Mining), information and semantic search in images and videos (Semantic Information Retrieval from images and videos), smart technology (Intelligent Technology), decision support systems in Smart Hospitals, Smart agriculture (Smart Algriculture), and smart environment (Smart Environment) based on Internet technology and objects (IoT) and cloud computing (Cloud Computing). Researches on Consulting Architecture Information System in e-Government (Enterprise Architecture in e-Government).

Some recent research directions:

Methods to support diagnosis of diseases from Medical images with Deep Learning.
Safety semi-supervised models applied in object detection on remote sensing images.
Transfer learning methods and knowledge graphs in disease detection and diagnosis
Medical Chatbot Systems
Technologies for extracting information on electronic medical records.
Large recommender system (Mega RS).
Facial expression analysis techniques.
Decision making models in credit scoring

Team

Ass. Prof. Dr. Le Hoang Son – Head of Department
Dr. Pham Huy Thong
Rohit Sharma – postdoctoral research (postdoc)
Luong Hong Lan – postdoctoral research (postdoc)
Le Minh Tuan – PhD student
Pham Hai Son – PhD student

Team of collaborators

Dr. Dinh Doan Long – University of Medicine and Pharmacy, VNU
Dr. Nguyen Thanh Tung – International School, VNU
Dr. Nguyen Long Giang – Academy of Science and Technology
Dr. Pham Van Hai – Hanoi University of Science and Technology
Nguyen Tran Quoc Vinh – University of Education, University of Danang
Dang Thanh Hai – Da Lat University
Dr. Huynh Xuan Hiep – Can Tho University
Dr. Vo Dinh Bay – Ho Chi Minh City University of Technology
Tran Manh Tuan – University of Water Resources
Dr. Tran Thi Ngan – University of Water Resources
Pham Minh Chuan – Hanoi National University of Education
Phung The Huan – University of Information Technology& TT, Thai Nguyen University
Cu Nguyen Giap – University of Commerce
Tran Thanh Dai – University of Economics – Industrial Technology
Adriana E. Chis – School of Computing, National University of Ireland
Horacio González-Vélez – School of Computing, National University of Ireland
Vassilis C. Gerogiannis – School of Technology, University of Thessaly, Greece
Anthony Karageorgos – School of Technology, University of Thessaly, Greece
Ganeshsree Selvachandran – UCSI University, Kuala Lumpur, Malaysia
Ishaani Priyadarshini – University of California, Berkeley, USA
Mumtaz Ali – Deakin University, Australia
Irfan Deli – Aralik Universitesi, Turkey
Sung Wook Baik – Sejong University, Korea
Shuo-Yan Chou – National Taiwan University of Science and Technology
Mr. Roberto Collonelo – Brainmatching srls, Italy
Hsu Hui Huang – Taipei Economic and Cultural Office in Vietnam
Angelo Ciaramella – Università degli Studi di Napoli Parthenope, Italy
Valentina Emilia Balas – Aurel Vlaicu Arad University, Romania
Raghvendra Kumar – GIET University, India

Some outstanding science and technology projects

The research staff of the Research Department of Multimedia Technology and Virtual Reality has carried out many projects at national, international and ministries, branches with results and potential applications. Here are some typical topics of the Department during the last 5 years (2017-2022):

Diagnostic software for periodontitis on dental X-ray images

Medical diagnosis is a very important step in the patient’s treatment process and is also a central step in clinical medicine. The correct diagnosis of the disease is an important requirement in making the right treatment decisions. A medical diagnosis is a prediction of a patient’s likelihood of disease based on information about the patient’s symptoms. Diagnosis is considered as the “backbone” of the medical industry, so ensuring and improving the quality of diagnosis is a matter of top concern in Vietnam as well as in the world. Today, information technology emerges as a bright spot in multidisciplinary cooperation. It is like a powerful tool to help improve performance and quality in many different areas of life. One of the fields that has reaped great achievements with the application of information technology is the health sector. In recent years, this multidisciplinary combination has reduced overcrowding in hospitals, improved medical services, quality of examination and treatment in hospitals, etc. Currently, it is very necessary to study the use of medical devices to support doctors’ diagnoses. To build this system, it is necessary to study and build a machine learning system to support the learning process to integrate into machine systems.

Originating from VNU-level project code QG.20.51, the software applies deep learning in image classification, improves accuracy, reduces processing time, with a friendly interface to help doctors diagnose X-ray image. The software has been granted a copyright certificate No. 7324/2021/QTG in 2021 from the Copyright Office.

Image: Diagnostic software from dental X-ray images

Facial morphology analysis software for Vietnamese people – VNCEPH

VNCEPH software has the function of 3D measurement, analysis and prediction of facial morphology in Vietnamese people, as a basis for research on pathology in Medicine. A software system to support the measurement of facial anthropometric indicators specifically for Vietnamese people needs to be built with specific tools to serve doctors and researchers in Vietnam. Based on the analysis of research, diagnosis and treatment needs of specialists, the measurement system and data storage on head-facial anthropometric indicators of Vietnamese people are gathered into a data warehouse. common data, unified. As a result, researchers in many different fields can use them under certain conditions to study the specific characteristics of the Vietnamese race.

The software is included in the results of the branch project of the Ministry of Science and Technology, code: ĐTL.CN.27/16, made in collaboration with the Institute of Odonto-Stomatology, Hanoi Medical University. and was granted a copyright certificate No. 5138/2017/QTG in 2017 from the Copyright Office.

Image: Software to measure maxillofacial anatomy

Research on building a consultation system and support diagnosis of diseases by fuzzy computing approach

Project under the National Foundation for Science and Technology Development (NAFOSTED) IN 2018, code: 102.05-2018.02 with the goal of focusing on research and development of new Soft Computing techniques, focus on fuzzy clustering techniques, recommender and decision support systems, and hybrid systems (Deep Learning, etc.) on advanced fuzzy sets (neutral fuzzy sets, etc.) in consulting and diagnostic support diagnose disease from medical imaging data (dental imaging test) and disease symptom data. The main research directions of the topic include:

+ Develop methods to represent, process and extract knowledge from medical databases:

For this content, the main approach is to represent knowledge according to fuzzy rules, decision trees, and modified neural networks. Fuzzy rules are expressed as IF THEN reflecting human knowledge in a particular field. Fuzzy rules can also be represented as decision trees or neural networks. In this part, fuzzy rules will be designed to be associated with the problem and medical data in the most effective way so that the process of retrieving the rule, determining the ‘closest’ rule (measurement of rule similarity), determining the rule Later contradictions are done quickly and accurately. This is important in the context of Big Data. Then the rule knowledge base (Knowledge) can be represented as XML specification language for easy access in diagnostic support and recommender systems. Another problem is that when merging multiple data sources (Data Fusion) leads to diverse rule systems. Then, fuzzy learning methods will be applied to improve system performance.

+ Proposing advanced fuzzy calculation methods in supporting disease diagnosis from image and symptom data.

The diagnostic process usually consists of 2 phases: identification of a potential group of diseases and identification of a list of diseases in that group. For the first task, advanced fuzzy set modeling approaches such as neutral fuzzy set (NS), picture fuzzy set (PFS), etc. are often used because of their ability to identify outliers, noises, defects, etc. through neutral memberships in NS or PFS. In medical image processing, areas that are less likely to have disease such as background areas are removed. In symptom data processing, a set of similar disease groups or similar symptom groups or similar patient groups is filtered out to serve as a reference basis for diagnosis. Then, in the 2nd task when identifying possible diseases, the set of rules built in content 1 will be used to determine the list of diseases from the image/symptoms. Besides, some machine learning methods such as K-neighbors, CNN networks can also be used for this task. Hybrid systems or granular computing are also used in evaluation and finding possible disease sets with different degrees of severity.

+ Building decision-making methods and recommender systems in diagnosis:

This problem is intended to solve the identification of the most likely disease in the list of diseases found in the previous content. This requirement refers to a decision making problem in which the evaluation criteria are in qualitative form, the number of evaluators can be many (consultations), etc. Approach to handling this requirement involves fuzzy classification and evaluation methods for multi-attribute (multi-criteria) decision making problems. Another issue that needs attention is building a Feedback Recommender System (Feedback Recommender System) to support interaction and predict future diseases (complications). The problem is that after diagnosis of disease A and treatment and re-diagnosis, a new list of diseases is obtained along with the results of the past (short). If at the time of the past, the patient had more than one disease, at the time of follow-up, the disease with the greatest likelihood of suffering from the two diseases was found in the past and present. Approaching feedback recommender systems with learning algorithms can help solve this problem.

The main results of the project have been published in 11 articles, including 01 prestigious ISI article, 06 prestigious international articles, 01 prestigious national paper, and 01 international paper. , and 02 National/International Conference papers. The project has trained 02 graduate students and 01 PhD student. The results of the project in addition to articles also build a strong research group, organize periodic seminars at the Institute of Information Technology, VNU and Thuy Loi University in the period of 2019-2021.

Figure: Main results of the project.

Some other typical topics have been and are being implemented

Research staff and collaborators of the department have also participated in implementing and chairing many research projects with good results, typically:

2022 – 2023: Research on some techniques to detect Diabetic Retinopathy from retinal images based on Deep Learning and Soft Computing models.
2022 – 2023: Research and develop group counseling techniques and applications in the selection of medical examination service packages.
2021 – 2022: Research and application of knowledge graphs in building an intelligent medical inquiry system.
2021 – 2022: Research and application of machine learning models in the analysis of patient electronic medical records of the gastrointestinal tract.
2020 – 2022: Application of artificial intelligence in flash flood analysis and warning, experimental for Lai Chau province.
2020 – 2022: Research and develop semi-supervised fuzzy clustering model and its application in decision support.
2020 – 2021: Research and develop reliable perspective fuzzy semi-supervised clustering algorithm and its application in forecasting.
2019 – 2020: Building a fuzzy learning system for application in medical diagnostic support
2018 – 2021: Research and development of particle computational model according to fuzzy set approach and application in dental diagnosis.
2017 – 2020: Research on building transfer learning models on complex fuzzy sets in decision support systems.

Some typical scientific works in the last 5 years

Phung The Huan, Pham Huy Thong, Tran Manh Tuan, Dang Trong Hop, Vu Duc Thai, Nguyen Hai Minh, Nguyen Long Giang, Le Hoang Son (2022), “TS3FCM: Trusted Safe Semi-Supervised Fuzzy Clustering Method for Data Partition with High Confidence”, Multimedia Tools and Applications, in press (SCIE, 2020 IF = 2.757), DOI = http://dx.doi.org/10.1007/s11042-022-12133-6
Ganeshsree Selvachandran, Shio Gai Quek, Luong Thi Hong Lan, Le Hoang Son, Nguyen Long Giang, Weiping Ding, Mohamed Abdel-Basset, Victor Hugo C. de Albuquerque (2021), “A New Design of Mamdani Complex Fuzzy Inference System for Multi-attribute Decision Making Problems”, IEEE Transactions on Fuzzy Systems, 29 (4), pp. 716 – 730 (SCI, 2020 IF = 12,029), DOI = http://dx.doi.org/10.1109/TFUZZ. 2019.2961350
Le Hoang Son, Roan Thi Ngan, Mumtaz Ali, Hamido Fujita, Mohamed Abdel-Basset, Nguyen Long Giang, Gunasekaran Manogaran, Priyan MK (2020), “A New Representation of Intuitionistic Fuzzy Systems and Their Applications in Critical Decision Making , IEEE Intelligent Systems, 35(1), pp. 6 – 17 (SCI, 2020 IF = 3.405), DOI = http://dx.doi.org/10.1109/MIS. 2019.2938441
Nguyen Long Giang, Le Hoang Son, Tran Thi Ngan, Tran Manh Tuan, Ho Thi Phuong, Mohamed Abdel-Basset, Antônio Roberto L. de Macêdo, Victor Hugo C. de Albuquerque (2020), “Novel Incremental Algorithms for Attribute Reduction from Dynamic Decision Tables using Hybrid Filter–Wrapper with Fuzzy Partition Distance”, IEEE Transactions on Fuzzy Systems, 28(5), pp. 858 – 873 (SCI, 2020 IF = 12,029), DOI = http://dx.doi.org/10.1109/TFUZZ. 2019.2948586
Roan Thi Ngan, Le Hoang Son, Mumtaz Ali, Dan E. Tamir, Naphtali D. Rishe, Abraham Kandel (2020), “Representing Complex Intuitionistic Fuzzy Set by Quaternion Numbers and Applications to Decision Making”, Applied Soft Computing, 87, pp. 105961 – 105976 (SCIE, 2020 IF = 6.725), DOI = https://doi.org/10.1016/j.asoc. 2019.105961.
Le Hoang Son, Hamido Fujita (2019), “Neural-Fuzzy with Representative Sets for Prediction of Student Performance”, Applied Intelligence, 49(1), pp. 172-187 (SCI, 2020 IF = 5.086), DOI = http://dx.doi.org/ 10.1007/s10489-018-1262-7
Nguyen Tho Thong, Luu Quoc Dat, Le Hoang Son, Nguyen Dinh Hoa, Mumtaz Ali, Florentin Smarandache (2019), “Dynamic Interval Valued Neutrosophic Set: Modeling Decision Making in Dynamic Environments”, Computers in Industry, 108, pp . 45 – 52 (SCIE, 2020 IF = 7,635), DOI = http://dx.doi.org/ 10.1016/j.compind.2019.02.009
Joshua Bapu, D. Jemi Florinabel, Y. Harold Robinson, E. Golden Julie, Raghvendra Kumar, Vo Truong Nhu Ngoc, Le Hoang Son, Tran Manh Tuan, Cu Nguyen Giap (2019), “Adaptive Convolutional Neural Network using N-gram for Spatial Object Recognition”, Earth Science Informatics, 12(4), pp. 525–540 (SCIE, 2020 IF = 2.878), DOI = http://dx.doi.org/ 10.1007/s12145-019-00396-x
Mumtaz Ali, Le Hoang Son, Mohsin Khan, Nguyen Thanh Tung (2018), “Segmentation of Dental X-ray Images in Medical Imaging using Neutrosophic Orthogonal Matrices”, Expert Systems With Applications, 91, pp. 434-441 (SCIE, 2020 IF = 6.954), DOI = http://dx.doi.org/ 10.1016/j.eswa.2017.09.027
Roan Thi Ngan, Mumtaz Ali, Le Hoang Son (2018), “d-Equality of Intuitionistic Fuzzy Sets: A New Proximity Measure and Applications in Medical Diagnosis”, Applied Intelligence, 48(2), pp. 499–525 (SCI, 2020 IF = 5.086), DOI = http://dx.doi.org/ 10.1007/s10489-017-0986-0.
Mumtaz Ali, Le Hoang Son, Nguyen Dang Thanh, Nguyen Van Minh (2018), “A Neutrosophic Recommender System for Medical Diagnosis Based on Algebraic Neutrosophic Measures”, Applied Soft Computing, 71, pp. 1054-1071 (SCIE, 2020 IF = 6.725), DOI = http://dx.doi.org/ 10.1016/j.asoc.2017.10.12

Multimedia and Virtual reality technology Division