Patient data analysis

IntermediateExploratory analysis (EDA)

An educational and research project only: its results are not a medical diagnosis and are not used to make any treatment decision.

Problem description

An exploratory analysis of health indicators for fictitious patients: detecting and imputing missing values and outliers, and studying the relationship between age and blood pressure. An educational and research project only: its results are not a medical diagnosis and must not be used for a treatment decision.

Objectives

  • Counting and handling missing values
  • Detecting outliers with the IQR method
  • Writing cautious conclusions

Problem type

Exploratory analysis (EDA)

Dataset

Tools used

Algorithms

Implementation steps

  1. In KNIME: File → Import KNIME Workflow… and choose patients-analysis.knwf, then Execute all.
  2. Open patients-analysis.ows in Orange.
  3. Review Column Statistics for the missing values.
  4. Use a Box Plot of the heart rate to see the outliers.
  5. Impute with the median, then Correlations.
  6. In KNIME: Missing Value → Numeric Outliers → Linear Correlation.

Workflow diagram

Orange

DataVisualize

File FileData Table Data TableColumn Statistics Column StatisticsBox Plot Box PlotDistributions DistributionsScatter Plot Scatter PlotImpute ImputeOutliers OutliersDataDataDataDataDataDataData
Drawn from the OWS workflow file attached to the project: the widgets, their icons, their category colours and the channel names on the links, as they appear when opened in Orange.

↔ Drag the diagram sideways to see all of it.

Connections as text (7)
  • File → Data Table (Data)
  • File → Column Statistics (Data)
  • File → Box Plot (Data)
  • File → Distributions (Data)
  • File → Scatter Plot (Data)
  • File → Impute (Data)
  • Impute → Outliers (Data)

KNIME

SourceManipulatorVisualizerOtherData portModel port

CSV Reader — SourceCSV ReaderNode 1Missing Value — ManipulatorMissing ValueNode 2Numeric Outliers — ManipulatorNumeric OutliersNode 3Linear Correlation — OtherLinear CorrelationNode 4Statistics — VisualizerStatisticsNode 5
KNIME nodes with their names in the node repository, their original icons, type colour, shape and ports (a triangle for data, a square for a model) and the status light, so you can build it in the same order. The ready KNWF workflow file is attached under "Project files", and it ran successfully in the KNIME 5.12 engine.

↔ Drag the diagram sideways to see all of it.

Connections as text (4)
  • CSV Reader → Missing Value
  • Missing Value → Numeric Outliers
  • Numeric Outliers → Linear Correlation
  • Missing Value → Statistics

Project files

The KNIME file (.knwf) was built on KNIME 5.12 and all its nodes ran successfully in the KNIME engine without opening the interface; open it from File → Import KNIME Workflow.

Results

Actual results computed on the attached files

Missing values: systolic_bp 7, glucose_mg_dl 18, bmi 10
IQR limits for the heart rate: from 50.5 to 102.5, and the number of readings outside them: 4
Correlation of age with systolic blood pressure: 0.654

Evaluation metrics

  • The number of missing values in each column
  • IQR limits
  • Correlation coefficient

Interpreting the results

The positive correlation between age and blood pressure was built into the data generation on purpose to teach reading results, and nothing medical can be concluded from it. Outliers may be entry errors or real cases, and the decision needs a domain expert. An educational and research project only: its results are not a medical diagnosis and must not be used for a treatment decision.

Discussion questions

  • When do you remove an outlier and when do you keep it?
  • Why is the median preferred to the mean for imputation here?
  • What are the ethical safeguards for handling real patient data?

Try it in the Data Preprocessing Lab

Similar projects