AI/ML Backend Engineer - LLM Data Labeling & Annotation
Managed end-to-end AI/ML model lifecycle tasks including data ingestion, supervised fine-tuning (SFT), preference modeling (RLHF), and model evaluation for large language models (LLM). Built Python NLP pipelines for preparing, curating, and processing both structured and unstructured training sets for model improvement. Orchestrated automated retraining, dataset versioning, benchmarking, and continuous learning workflows to optimize model performance using feedback loops. • Designed data curation workflows for RLHF, including annotation and reward signal optimization • Performed prompt engineering and scoring for SFT data collection and generation • Evaluated model outputs on diverse datasets (code, natural language) using SWE-Bench, OSWorld, and Terminal Bench • Logged, analyzed, and iteratively updated dataset labels, predictions, and feedback using ELK Stack and GCP monitoring tools