Software Development Engineer in Test
11 days ago
Orlando
Job DescriptionSoftware Development Engineer in Test (SDET) – AI/ML Quality Assurance Overview We are seeking a highly skilled Software Development Engineer in Test (SDET) with deep experience in AI/ML systems. This role focuses on advancing the reliability, safety, and performance of cutting-edge generative AI and traditional ML models. The ideal candidate will have strong software engineering fundamentals, proven experience with automated testing, and specialized expertise in validating AI/ML models and pipelines. Job at a Glance • Job Title: Software Development Engineer in Test (SDET) – AI/ML, • Location: Orlando, FL; Seattle, WA; or Glendale, CA, • Schedule: Hybrid (on-site 3 days per week), • Contract Type: 22-month initial contract with potential conversion to full-time, • Design, build, and maintain automated testing frameworks for backend, frontend, and ML components., • Validate and define acceptance criteria in BDD/Gherkin for AI/ML features., • Develop and execute test strategies for generative AI models, traditional ML models, agents, and knowledge base integrations., • Implement automated testing for ML performance metrics (accuracy, precision, recall, F1-score, and business-defined KPIs)., • Create frameworks for validating ML pipelines, data preprocessing, feature engineering, training, and inference., • Design and execute A/B testing frameworks for comparing model performance and outputs.AI Safety & Guardrails, • Evaluate AI/ML outputs for hallucinations, bias, and accuracy using leading evaluation frameworks., • Implement guardrails and perform adversarial/red team testing to ensure safe and responsible AI behavior., • Develop testing protocols for fairness, interpretability, explainability, and compliance., • Build automated detection systems for model/data drift and performance degradation.Automation & Engineering, • Take ownership over test automation, designing and customizing tools as needed., • Develop feature code and support engineering efforts outside of test deliverables., • Quickly translate technical requirements into automated test coverage., • Integrate testing into ML deployment pipelines and model lifecycle management., • Support CI/CD pipelines, including Jenkins, Xcode Server, and VS Build Server, to manage test targets and reporting., • Participate in code reviews, technical specification development, and agile sprint planning.Collaboration & Leadership, • Partner with ML engineers, data scientists, and product managers to drive quality across the ML lifecycle., • Collaborate with QA and Performance Engineering teams to validate AI/ML workloads under load and stress conditions., • Influence SDLC processes through best practices in testing, automation, and quality engineering., • Bachelor’s degree in Computer Science, Software Engineering, Electrical/Electronics Engineering, or equivalent experience., • Proven experience as an SDET with AI/ML focus or as a Machine Learning/AI Engineer with strong testing expertise., • Track record testing supervised, unsupervised, reinforcement learning, and generative AI models., • Proficiency in Python and ML libraries such as PyTorch, TensorFlow, Pandas, Scikit-learn, NumPy, and Matplotlib., • Hands-on experience with ML model deployment, validation, monitoring, and maintenance., • Deep knowledge of modern testing methodologies for software and ML systems., • Experience with ML model lifecycle management, versioning, and A/B testing frameworks., • Proficiency with cloud ML platforms: GCP (Vertex AI, BigQuery ML), Azure (Azure ML, Cognitive Services), or AWS (SageMaker, Bedrock)., • Hands-on experience with CI/CD pipelines and MLOps practices., • Familiarity with REST API testing (Postman), database testing (SQL), and data validation for ML training sets., • Experience with ML interpretability tools (SHAP, LIME) and monitoring tools (MLflow, Weights & Biases)., • Familiarity with model observability, feature store validation, and distributed ML training/inference., • Knowledge of ML security, privacy, and compliance testing., • iOS development or QE (Quality Engineering) background., • Passion for advancing AI/ML safety and reliability in real-world production systems. #ZR #INDGEN