A prototype medical decision support system that utilizes 'sensitivity' and 'specificity' of a clinical feature for diagnosis: a novel approach

A prototype computer based decision support system was developed to simulate a doctors' decision making process using a relational database consisting 25 clinical features and 10 common diseases encountered in critical care. The relationship between diseases and clinical features was cited by a sensitivity and a specificity value for each clinical feature. A clinical expert arbitrarily determined the sensitivity and specificity values. The cumulative probability values of each disease in relation to presenting clinical features were calculated using simple decision algorithm with ranked values to determine the most probable diagnosis. The database was built using Microsoft Access and the interfaces in Visual Basic environment. In the program the output window provides the user with 5 most likely diagnoses with a display of ranked probability values. This differential diagnosis can be refined repetitively using new information. The system was validated using data from 26 patients admitted to a regional intensive care unit. The prototype decision support system was able to predict the true diagnosis with a sensitivity value of 88% as rank 1 and 96% as both rank 1 or 2. Thus results show that this novel approach of decision support could be more reliable to assist a doctor.


Introduction
The use of computers in medicine is on the increase to manage large amount of information and to make evidence based and cost effective decisions on a daily basis(l). Already the computer aided medical tools assist in managing the growing information needs of the busy clinicians and improve healthcare processes as well as patient outcomes (2). This process has led to rapid proliferation of computerized clinical decision support systems (CDSS)(3). The use of CDSS systems has convincingly shown changes in physician behavior and improved patient outcomes (4). Different approaches have been adopted to develop CDSS s ranging from simple mathematics to artificial intelligence techniques such as neural networks and fuzzy logic(4).
A CDSS is a computer based tool, which uses an explicit knowledge to generate patient specific advice or interpretation (5). What is ultimately needed is to simulate the complex decision making process of a medical officer investigating a patient. In clinical practice, a doctor would first decide on a differential diagnosis observing the presenting clinical features. Thereafter, he continues to review and investigate the patient to recognize new and more specific clinical features to sharpen the focus on a final diagnosis. This type of homing on a specific diagnosis cannot be carried out by logical deduction. Thus, our approach in this research was to use a probabilistic decision making approach, utilizing a rank, which can be refined repetitively as new information becomes available.

Objectives
The main objectives of our investigation were, to (a) design and construct a prototype medical database suitable for the proposed CDSS, (b) develop a preliminary decision algorithm for the decision making process and (c) validation of the decision support system using real clinical data.

Methods
The structure for the main database was designed and implemented using Microsoft Access. This database consists of four basic tables. The disease table contained 10 common diseases encountered in critical care, namely, heart failure, septic shock, pneumonia, asthma, renal failure, hepatitis, myocardial infarction, organophosphate (OP) poisoning, epilepsy and pre-eclampsia, tagged with a unique identification code. The clinical feature table had symptoms, signs and investigation results such as cough, breathlessness, chest pain, fever, ECG with ischaemic features, albuminuria, pinpoint pupils, cyanosis, jaundice, convulsions, altered consciousness, cold peripheries, tachycardia, high blood pressure, low blood pressure, high blood urea, high blood creatinine, cardiomegaly on chest X-ray, collapse/consolidation on chest X-ray, pregnancy, wheezing, thrombocytopenia, low urine output, OP smell and liver tenderness in a coded format. The two statistical quantities, namely 'sensitivity' and 'specificity' of each clinical feature against each disease formed the third or the main table. The fourth table contained patients' data and their clinical features entered into the computer by the users.
The main data table had four fields. They were disease code, clinical feature code and values of sensitivity and specificity. When a new disease or a clinical feature was added through the user interface, the main data table was updated automatically and coded. The main data-editing window also provided an editing facility to directly enter sensitivity, specificity values if known. Above data constituted the expert knowledge of the system. The editing process was password protected to prevent corruption of the database by unauthorized users.

Clinical Validation
The computer program was tested retrospectively utilizing clinical information of 26 patients who were diagnosed to suffer from the diseases incorporated in the database and were receiving treatment at the Intensive Care Unit, Teaching Hospital, Peradeniya. The doctors ultimate diagnosis was compared with the ranked diagnoses obtained via the CDSS for each patient after feeding the CDSS with patients real clinical features at presentation. The ability to pickup the doctor's diagnosis by the CDSS was expressed as a sensitivity value.

Results
The decision making module of this prototype CDSS with its decision algorithm and adaptive mechanism is shown in Figure 1.
The system is menu driven and Figure 2 shows the clinical feature input interface. Clinical features of the patient are selected from a drop down list.
The "Process Diagnosis" button activates the output and is shown in Figure 3. This can be refined using "Refine diagnosis" which activates the interface shown in Figure 4. Table 1 shows the clinical diagnosis reached by the CDSS against the true diagnosis. Twenty three of the 26 cases were correctly diagnosed by the system as the first rank. Two of the 26 cases were correctly diagnosed in the 2 nd rank.  The validation of the prototype system has shown that the CDSS predicts the diagnosis of diseases as first rank with 88% sensitivity. Because the doctors' final diagnosis was considered the gold standard in this retrospective study, there were no negatively diagnosed cases. Hence the specificity value of the CDSS could not be evaluated. This current performance can be further improved by adaptive mechanisms, which will eventually fine-tune the system with its own experience. Even with a few sets of clinical features and diseases the prototype system appear to function reliably to achieve a probability based ranked differential diagnosis.

Clinical validation
The current database and the user interface were made to facilitate the preliminary study of this project. Hitherto we have dealt with only a small quantity of data and hence the data retrieval process is not critical. However, as the database expands it would be necessary not only to restructure the database, but also to improve the retrieval techniques.
As shown in Figure 1, when there is an error (or mismatch) between the estimated disease and the actually diagnosed disease (diagnosed by an expert), a mechanism is required to process this information to upgrade the parameters involved in the decision algorithm, in particular, the sensitivity and specificity data. We are aware that a doctor based in a tropical country may act differently to a doctor based in a sub tropical country given the same set of clinical features. This is because there is a difference in the prevalence in diseases between different regions and countries. These regional and geographical variations also need to be accounted for in advanced systems(4).
In summary, this CDSS could serve medical subspecialties by simply changing-the database appropriately. We consider this novel concept may be a realistic approach for artificial intelligence in medicine because of its features listed below.

The concepts of sensitivity and
specificity relating the clinical features to diseases in fact form out knowledge base. These quantities are obtained initially from the experts and thereafter through the self learning mechanism of the system.

2.
The decision rules of this system are appropriately learned mappings relating the respective sensitivity and specificity terms invoked by the observed clinical features to the diseases with a certainty ranking. However these mappings, which are probably non-linear, have to be further investigated.
3. By giving a ranked diagnosis as well as the option for re-entering new data for the most specific and important clinical features of the each ranked disease, this system also provides an iterative diagnostic process to the user, which in fact mimics the doctors' diagnostic approach.
4. Finally and more importantly, the proposed adaptive mechanism would not only update the functional parameters in 2., but would also improve the sensitivity and specificity.
With these four features combined, this proposed mechanism is expected to have the decision making capability, simulating the doctors' iterative approach for clinical diagnosis.