RESPONSIBLE RISK ASSESSMENT OF MACHINE LEARNING TOOLS FOR PROMOTION BOARDS

This thesis evaluates an artificial intelligence (AI) model's ability to support U.S. Marine Corps promotion boards, investigating how data drift and feedback loops impact its performance and reliability over time. Using the DataRobot automated machine learning (AutoML) platform, a model was trained on fiscal year (FY) 2023 Major promotion board data and subsequently tested against FY24 and FY25 data to measure performance degradation and detect statistical drift in key predictors. The model demonstrated strong but brittle performance, maintaining high ranking accuracy across years while relying heavily on administrative features like promotion zone, which exhibited significant data drift. The research concludes that without robust governance, the model's dependence on procedural proxies risks amplifying biases and capturing administrative timing rather than a holistic view of merit, potentially undermining the promotion system's integrity. It is recommended that the Marine Corps use the AI tool for pre-screening to improve efficiency, while implementing annual retraining, continuous drift monitoring, and explicit fairness thresholds to ensure long-term effectiveness and trust.