healthcare

Introduction:

COVID-19 was first recognized on December 30, 2019, by BlueDot. It did so nine days before the World Health Organization released its alert for coronavirus. How did BlueDot do it? BlueDot used the power of AI and data science to predict and track infectious diseases. It identified an emerging risk of unusual pneumonia happening around a market in Wuhan.

The role of data science and AI in the Healthcare industry is not limited to that. Now, it has become possible to learn the causes of whatever symptoms you are experiencing, such as cough, fever, and body pain, without visiting a doctor and self-treating it at home. Platforms like Ada Health and Sensely can diagnose the symptoms you report.

The Healthcare industry generates 30% of 1.145 trillion MB of data generated every day. This enormous amount of data is the driving force for revolutionizing the industry and bringing convenience to people’s lives.

Applications of Data Science in Healthcare:

1. Prediction and spread of diseases

Predictive analysis, using historical data to find patterns and predict future outcomes, can find the correlation between symptoms, patients’ habits, and diseases to derive meaningful predictions from the data. Here are some examples of how predictive analytics plays a role in improving the quality of life and medical condition of the patients:

According to a survey carried out in January 2020, 85 percent of the respondents working in smart hospitals reported being satisfied with their work compared to 80 percent of the respondents from digital hospitals. Similarly, 74 percent of the respondents from smart hospitals would recommend the medical profession to others, while only 66 percent of the respondents from digital hospitals recommend it.

Staff retention has been a challenge but is now becoming an enormous challenge, especially post-pandemic. For instance, after six months of the COVID-19 outbreak, almost a quarter of care staff quit their job in Flanders & Belgium. The care staff felt exhausted, experienced sleep deprivation, and could not relax properly. A Smart healthcare system can solve these issues.

Smart healthcare systems can help optimize operations and provide prompt service to patients. It forecasts the patient load at a particular time and plans resources to improve patient care. It can optimize clinic staff scheduling and supply, which reduces the waiting time and overall experience.

Getting data from partners and other third-party sources can be beneficial too. Data from various sources can help in process management, real-time monitoring, and operational efficiency. It leads to overall clinic performance optimization. We can perform deep analytics of this data to make predictions for the next 24 hours, which helps the staff focus on delivering care.

3. Data science for medical imaging

According to the World Health Organization (WHO), radiology services are not accessible to two-thirds of the world population. Patients must wait for weeks and travel distances for simple ultrasound scans. One of the foremost uses of data science in the healthcare industry is medical imaging. Data Science is now used to inspect images from X-rays, MRIs, and CT scan to find irregularities. Traditionally, radiologists did this task manually, but it was difficult for them to find microscopic deformities. The patient’s treatment depends highly on insights gained from these images.

Data science can help radiologists with image segmentation to identify different anatomical regions. Applying some image processing techniques like noise reduction & removal, edge detection, image recognition, image enhancement, and reconstruction can also help with inspecting images and gaining insights.

One example of a platform that uses data science for medical imaging is Medo. It provides a fully automated platform that enables quick and accurate imaging evaluations. Medo transforms scans taken from different angles into a 3D model. They compare this 3D model against a database of millions of other scans using machine learning to produce a recommended diagnosis in real-time. Platforms like Medo make radiology services more accessible to the population worldwide.

4. Drug discovery with data science

Traditionally, it took decades to discover a new drug, but the time has now been reduced to less than a year using data science. Drug discovery is a complex task. Pharmaceutical industries rely heavily on data science to develop better drugs. Researchers need to identify the causative agent and understand its characteristics which may require millions of test cases to understand. This is a huge problem for pharmaceutical companies because it can take decades to perform these tests. Data science solved this problem and can perform this task in a month or even a few weeks.

For example, the causative agent for COVID-19 is the SARS-CoV-2 virus. For discovering an effective drug for COVID-19, deep learning is used to identify and design a molecule that binds to SARS-CoV-2 to inhibit its function by using extracted data from scientific literature through NLP (Natural Language Processing).

5. Monitoring patients’ health

The human body generates two terabytes of data daily. Humans are trying to collect most of this data using smart home devices and wearables. The data these devices collect includes heart rate, blood sugar, and even brain activity. Data can revolutionize the healthcare industry if known how to use it.

Every 36 seconds, a person dies from cardiovascular disease in the United States. Data science can identify common conditions and predict disorders by identifying the slightest change in the health indicators. Timely alert of changes in health indicators can save thousands of lives. Personal health coaches are designed to help to gain deep insights into the patient’s health and alert if the health indicator reaches a dangerous level.

Companies like Corti can detect cardiac arrest in 48 seconds through phone calls. This solution uses real-time natural language processing to listen to emergency calls and look out for several verbal and non-verbal patterns of communication. It is trained on a dataset of emergency calls and acts as a personal assistant of the call responder. It helps the responder ask relevant questions, provide insights, and predict if the caller is suffering from cardiac arrest. Corti finds cardiac arrest more accurately and faster than humans.

6. Virtual assistants in healthcare

The WHO estimated that by 2030, the world will need an extra 18 million health workers worldwide. Using virtual assistant platforms can fulfill this need. According to a survey by Nuance, 92% of clinicians believe virtual assistant capabilities would reduce the burden on the care team and patient experience.

Patients can enter their symptoms as input to the platform and ask questions. The platform would tell you about your medical condition using the data of symptoms and causes. It is possible because of the predictive modeling of disease. These platforms can also assist patients in many other ways, like reminding them to take medication on time.

An example of such a platform is Ada Health, an AI-enabled symptom checker. A person enters symptoms through a chatbot, and Ada uses all available data from patients, past medical history, EHR implementation, and other sources to predict a potential health issue. Over 11 million people (about twice the population of Arizona) use this platform.

Other examples of health chatbots are Babylon Health, Sensely, and Florence.

Conclusion:

In this blog, we discussed the applications of AI in healthcare. We learned about some businesses and startups that are using AI to revolutionize the healthcare industry. This advancement in AI has helped in fighting against Covid19. To learn more about data science enroll in our Data Science Bootcamp, a remote instructor-led Bootcamp where you will learn data science through a series of lectures and hands-on exercises. Next, we will be creating a prognosis prediction system in python. You can follow along with my next blog post here.

Want to create data science applications with python? checkout our Python for Data Science training. 

Introduction to prognosis prediction

Import required modules

Let us start by importing all the libraries needed in the exercise. We import pandas as we will be reading CSV files as Data Frame. We are importing Label Encoder from sklearn.preprocessing package. Label Encoder is a utility class to convert non-numerical labels to numerical labels. In this exercise, we predict prognosis using symptoms, so it is a classification task.

We are using RandomForestClassifier, which consists of many individual decision trees that work as an ensemble. Learn more about RandomForestClassifier by enrolling in our Data Science Bootcamp, a remote instructor-led Bootcamp. We also require classification reports and accuracy score metrics to measure the model’s performance.

import numpy as np
import pandas as pd
from sklearn.preprocessing import LabelEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, accuracy_score

Read CSV files

We are using this Kaggle dataset for our exercise.

It has two files, Training.csv and Testing.csv, containing training and testing data, respectively. You can download these files by going to the data section of the above link.

Read CSV files into Data Frame using pandas read_csv() function. It reads comma-separated files at supplied file path into DataFrame. It takes a file path as a parameter, so provide the right file path where you have downloaded the files.

train = pd.read_csv("File path of Training.csv")
test = pd.read_csv("File path of Testing.csv")

Check samples of the training dataset

To check what the data looks like, let us grab the first five rows of the DataFrame using the head() function.

We have 133 features. We want to predict prognosis so that it would be our target variable. The rest of the 132 features are symptoms that a person experience. The classifier would use these 132 symptoms feature to predict prognosis.

train.head()

The training set holds 4920 samples and 133 features, as shown by the shape attribute of the DataFrame.

train.shape

Output
(4920, 133)

Descriptive analysis

Description of the data in the DataFrame can be seen by describe() method of the DataFrame. We see no missing values in our DataFrame as the count of all the features is 4920, which is also the number of samples in our DataFrame. We also see that all the numeric features are binary and have a value of either 1 or 0.

train.describe()

train.describe(include=['object'])

Our target variable prognosis has 41 unique values, so there are 41 diseases in which the model will classify input. There are 120 samples for each unique prognoses in our dataset.

train['prognosis'].value_counts()

There are 132 symptoms in our dataset. The names of the symptoms will be listed if we use this code block.

possible_symptoms = train[train.columns.difference(['prognosis'])].columnsprint(list(possible_symptoms))

Output
['abdominal_pain', 'abnormal_menstruation', 'acidity', 'acute_liver_failure', 'altered_sensorium', 'anxiety', 'back_pain', 'belly_pain', 'blackheads', 'bladder_discomfort', 'blister', 'blood_in_sputum', 'bloody_stool', 'blurred_and_distorted_vision', 'breathlessness', 'brittle_nails', 'bruising', 'burning_micturition', 'chest_pain', 'chills', 'cold_hands_and_feets', 'coma', 'congestion', 'constipation', 'continuous_feel_of_urine', 'continuous_sneezing', 'cough', 'cramps', 'dark_urine', 'dehydration', 'depression', 'diarrhoea', 'dischromic _patches', 'distention_of_abdomen', 'dizziness', 'drying_and_tingling_lips', 'enlarged_thyroid', 'excessive_hunger', 'extra_marital_contacts', 'family_history', 'fast_heart_rate', 'fatigue', 'fluid_overload', 'fluid_overload.1', 'foul_smell_of urine', 'headache', 'high_fever', 'hip_joint_pain', 'history_of_alcohol_consumption', 'increased_appetite', 'indigestion', 'inflammatory_nails', 'internal_itching', 'irregular_sugar_level', 'irritability', 'irritation_in_anus', 'itching', 'joint_pain', 'knee_pain', 'lack_of_concentration', 'lethargy', 'loss_of_appetite', 'loss_of_balance', 'loss_of_smell', 'malaise', 'mild_fever', 'mood_swings', 'movement_stiffness', 'mucoid_sputum', 'muscle_pain', 'muscle_wasting', 'muscle_weakness', 'nausea', 'neck_pain', 'nodal_skin_eruptions', 'obesity', 'pain_behind_the_eyes', 'pain_during_bowel_movements', 'pain_in_anal_region', 'painful_walking', 'palpitations', 'passage_of_gases', 'patches_in_throat', 'phlegm', 'polyuria', 'prominent_veins_on_calf', 'puffy_face_and_eyes', 'pus_filled_pimples', 'receiving_blood_transfusion', 'receiving_unsterile_injections', 'red_sore_around_nose', 'red_spots_over_body', 'redness_of_eyes', 'restlessness', 'runny_nose', 'rusty_sputum', 'scurring', 'shivering', 'silver_like_dusting', 'sinus_pressure', 'skin_peeling', 'skin_rash', 'slurred_speech', 'small_dents_in_nails', 'spinning_movements', 'spotting_ urination', 'stiff_neck', 'stomach_bleeding', 'stomach_pain', 'sunken_eyes', 'sweating', 'swelled_lymph_nodes', 'swelling_joints', 'swelling_of_stomach', 'swollen_blood_vessels', 'swollen_extremeties', 'swollen_legs', 'throat_irritation', 'toxic_look_(typhos)', 'ulcers_on_tongue', 'unsteadiness', 'visual_disturbances', 'vomiting', 'watering_from_eyes', 'weakness_in_limbs', 'weakness_of_one_body_side', 'weight_gain', 'weight_loss', 'yellow_crust_ooze', 'yellow_urine', 'yellowing_of_eyes', 'yellowish_skin']

There are 41 unique prognoses in our dataset. The name of all prognoses will be listed if we use this code block:

list(train['prognosis'].unique())

Output
['Fungal infection','Allergy','GERD','Chronic cholestasis','Drug Reaction','Peptic ulcer diseae','AIDS','Diabetes ','Gastroenteritis','Bronchial Asthma','Hypertension ','Migraine','Cervical spondylosis','Paralysis (brain hemorrhage)','Jaundice','Malaria','Chicken pox','Dengue','Typhoid','hepatitis A','Hepatitis B','Hepatitis C','Hepatitis D','Hepatitis E','Alcoholic hepatitis','Tuberculosis','Common Cold','Pneumonia','Dimorphic hemmorhoids(piles)','Heart attack','Varicose veins','Hypothyroidism','Hyperthyroidism','Hypoglycemia','Osteoarthristis','Arthritis','(vertigo) Paroymsal  Positional Vertigo','Acne','Urinary tract infection','Psoriasis','Impetigo']

Data visualization

new_df = train[train.columns.difference(['prognosis'])]
#Maximum Symptoms present for a Prognosis are 17
new_df.sum(axis=1).max()
Minimum Symptoms present for a Prognosis are 3
new_df.sum(axis=1).min()
series = new_df.sum(axis=0).nlargest(n=15)
pd.DataFrame(series, columns=["Occurance"]).loc[::-1, :].plot(kind="barh")

Fatigue and vomiting are the symptoms most often seen.

Encode object prognosis

Our target variable is categorical features. Let us create an instance of Label Encoder and fit it with the prognosis column of train data and test data. It will encode all possible categorical values in numerical values.

label_encoder = LabelEncoder()
label_encoder.fit(pd.concat([train['prognosis'], test['prognosis']]))

It concludes the data preparation step. Now, we can move on to model training with this data.

Training and evaluating model

Let us train a RandomForestClassifier with the prepared data. We initialize RandomForestClassifier, fit the features and label in it then finally make a prediction on our test data.

In the end, we transform label encoded prognosis values back to the original form using the fit_transform() method of the LabelEncoder object.

random_forest = RandomForestClassifier()
random_forest.fit(train[train.columns.difference(['prognosis'])], label_encoder.fit_transform(train['prognosis']))
y_pred = random_forest.predict(test[test.columns.difference(['prognosis'])])
y_true = label_encoder.fit_transform(test['prognosis'])
print("Accuracy:", accuracy_score(y_true, y_pred))
print(classification_report(y_true, y_pred, target_names=test['prognosis']))

Predict prognosis by taking symptoms as input

We have our model trained and ready to make predictions. We need to create a function that takes symptoms as input and predicts the prognosis as output. The function predict_prognosis() below is just doing that.

We take input features as a string of symptoms separated by space. We strip the string to remove spaces at the beginning and end of the string. We split this string and created a list of symptoms. We cannot use this list directly in the model for prediction as it contains symptoms’ names, but our model takes a list of 0 and 1 for the absence and presence of symptoms. Finally, with the features in the desired form, we predict the prognosis and print the predicted prognosis.

def predict_prognosis():
  print("List of possible Symptoms you can enter: ", list(train[train.columns.difference(['prognosis'])].columns))
  input_symptoms = list(input("\nEnter symptoms space separated: ").strip().split())
  print(input_symptoms)
  test_value = []
  for symptom in train[train.columns.difference(['prognosis'])].columns:
    if symptom in input_symptoms:
      test_value.append(1)
    else:
      test_value.append(0)
    np_test = np.array(test_value).reshape(1, -1)
    encoded_label = random_forest.predict(np_test)
  predicted_label = label_encoder.inverse_transform(encoded_label)[0]
  print("Predicted Prognosis: ", predicted_label)
predict_prognosis()

Give input symptoms:

Conclusion

To sum up, we discussed the applications of AI in healthcare. Took a deep dive into an application of AI, and prognosis prediction using an exercise. Created a prognosis predictor with an explanation of each step. Finally, we tested our predictor by giving it input symptoms and got the prognosis as output.