Drug Transporter ML

This repository is the machine learning implementation behind a published, peer-reviewed paper: Nigam AK, Li JG, Lall K, Shi D, Bush KT, Bhatnagar V, Abagyan R, Nigam SK, "Unique metabolite preferences of the drug transporters OAT1 and OAT3 analyzed by machine learning," Journal of Biological Chemistry, 2020 (PMID 31896576, DOI 10.1074/jbc.RA119.010729). It attacks one classification problem three different ways: exhaustive brute-force search, a NEAT-evolved network used directly as a classifier, and a NEAT-evolved network used as a feature transformer ahead of a Random Forest, asking whether letting evolution search the model space beats searching the feature space directly.

This case study is based on running the code, not just reading it. The brute-force path works exactly as documented: it loads the real 51-sample and 73-sample datasets, runs 10-fold cross-validated Random Forest and Decision Tree classifiers, and produces real accuracy numbers on live data. The two NEAT-based paths, as currently packaged, do not run: the bundled config files are missing a parameter, no_fitness_termination, that current neat-python versions require, so both feature_selection_GA.py and feature_engineering_GA.py crash on the first line of setup. The exact cause was diagnosed and the one-line fix confirmed in a scratch copy, with a correct evolutionary run reproduced once patched. The tracked repo itself still needs that fix before its own README instructions work end to end.

OAT1 (SLC22A6) and OAT3 (SLC22A8) are kidney transporters that move small organic anions, including a large share of prescribed drugs and their metabolites, out of the blood for elimination. Which transporter handles a given compound affects dosing, drug-drug interaction risk, and nephrotoxicity screening. Given a molecule's physicochemical descriptors, molecular weight, volume, LogP, LogS, polar surface area, ring counts, charge counts, 32 features per compound in total, can you predict whether it is an OAT1 substrate, an OAT3 substrate, both, or neither, well enough to say something about which molecular properties actually drive the preference? The repo packages the ML side of that work: three metabolite datasets and three independent algorithms for finding the feature subsets and models that separate these classes.

Three datasets, each a CSV of molecules with 32 shared descriptor columns and a transporter-class label: OAT1OAT3Small.csv (51 metabolites, 3-class: 17 Both, 12 OAT1-only, 22 OAT3-only), OAT1OAT3Big.csv (73 metabolites, same 3-class scheme: 17 Both, 21 OAT1, 35 OAT3), and OAT1OAT3OATP.csv (137 metabolites combining kidney and liver transporters, 6-class: 73 kidney rows plus 64 liver rows split 29 Both OATP, 20 OATP1B1, 15 OATP1B3).

Three algorithms run against any of the three datasets. Brute force exhaustively evaluates every size-k combination of the 32 features, training a fresh Random Forest and Decision Tree on each combination with 10-fold cross-validation, and writes every combination's accuracy to CSV, the "ground truth" search if you are willing to pay for C(32, k) model fits. Feature selection GA uses NEAT (NeuroEvolution of Augmenting Topologies) to evolve a small neural network that takes all 32 raw features and outputs a class score directly, fitness defined as training-set classification accuracy, replacing the classifier itself with an evolved network. Feature engineering GA also uses NEAT, but the evolved network transforms the 32 raw features into 6 new engineered features, scored by a downstream Random Forest under cross-validation, closer to automated feature construction than classification.

All three share a data loader for CSV loading and cleaning and a small shared utility module for the accuracy metric, NEAT population bootstrapping, and CLI prompts. An interactive CLI ties it together: pick a dataset, pick an algorithm, run. A legacy/ directory holds six earlier implementations across four conceptual iterations, kept rather than deleted, an honest record of how the analysis evolved from a five-classifier brute force to NEAT.

NEAT over gradient descent was a reasonable call for this scale: with 32 features and 51 to 137 samples, a hand-rolled backprop net would need real regularization to avoid memorizing the training set, and NEAT starts from minimal topology and only grows structure that improves fitness. The three algorithms are not evaluated the same way, though, and that matters if you are comparing their numbers. The feature-selection GA's docstring says outright that it "uses the entire thing to train and uses training accuracy as a measure of fitness," no held-out split. The feature-engineering GA does better, scoring its downstream Random Forest with leave-one-out or 10-fold CV depending on dataset size. Brute force is the most rigorous of the three, every combination gets 10-fold CV, no shortcuts. Winner-fitness numbers across the three methods are not directly comparable.

A few smaller things surfaced while running the code. nCr() uses floating-point division purely to size a progress bar. It printed 495.99999999999994 instead of 496 for C(32, 2), cosmetic only, since the actual search is driven by itertools.combinations directly. utilities.py hardcodes a Windows Graphviz path into PATH at import time. It will not crash elsewhere, but network-visualization rendering will silently fail to find the dot executable on any other machine. There are no automated tests, defensible for research code built to produce one paper's result and then left alone, but the tradeoff shows up later: the neat-python config incompatibility documented below would have been caught immediately by even a minimal smoke test, and instead sat undiscovered until this audit.

Verified live during this audit, with pandas 2.3.3, numpy 1.26.4, scikit-learn 1.8.0, and neat-python 2.0.0. Data loading is correct: all three CSV loaders return exactly the shapes the code implies, 51x32 with 3-class labels (17 Both, 12 OAT1, 22 OAT3), 73x32 (17 Both, 21 OAT1, 35 OAT3), and 137x32 with 6-class labels split 73 kidney and 64 liver rows.

Brute force runs correctly end to end: 5 of the 496 possible 2-feature combinations from the small dataset were run through the real classifier pipeline, producing accuracies from 0.34 to 0.42 (Random Forest) and 0.40 to 0.44 (Decision Tree) against a majority-class baseline of 43.1%. That is a small, non-exhaustive slice, not the full sweep, so it says nothing about the paper's actual best-feature-subset result. It confirms the mechanism works and produces sane numbers on real data.

Both NEAT-based algorithms fail immediately on a stock checkout: running either one against any of the four bundled config files raises a RuntimeError for a missing required configuration item, no_fitness_termination, that the neat-python 2.0.0 config schema requires but none of the four files provide. The fix is real and small: adding one line under [NEAT] in a scratch copy, never the tracked file, let the feature-selection GA run correctly, population fitness climbing from 0.451 at generation 0 to 0.569 at generation 4 over a 5-generation test run. A second, separate issue then surfaced: NEAT's checkpoint writer targets ./output/, which does not exist in a fresh checkout, until the directory is created by hand.

Brute force is only tractable because k stays small. C(32, 2) is 496 combinations, but C(32, 6) is over 900,000, each needing two classifiers fit under 10-fold CV. The exhaustive-search guarantee only holds for the small-k regime the paper actually used. The CLI is interactive-only, a while True loop of input() prompts, fine for one person running experiments by hand, but there is no way to script a full sweep across datasets, algorithms, and seeds without hacking around stdin.

The strongest lesson from this audit is about maintenance debt in research code: a dependency floor with no upper bound and no CI meant a config-schema change three major versions upstream silently broke two of the three documented entry points, and nobody would know until they actually tried to run them, since the README's instructions look complete and correct on paper. The brute-force path, which happens to be the one that does not touch NEAT config parsing at all, is the only one that still works untouched. That is not a flaw in the original 2020 work so much as what happens to any unmaintained research repo left alone for years against a fast-moving dependency. The fix is one line per config file, but finding it required actually running the code, not just reading it.