Full Stack Engineer
hace 2 días
Chicago
Candidate must already live in Chicago! No exceptions! \n \n Senior Full Stack Engineer \n Contract Position \n Position Overview \n We are seeking an experienced Senior Full Stack & Cloud-Native AI Engineer to transform an existing Data Insight proof of concept into a secure, scalable, production-ready enterprise application. \n The application enables users to explore reporting data, ask contextual follow-up questions, and gain actionable insights supported by verifiable source evidence. \n In this role, you will design and develop end-to-end application capabilities, including agentic AI workflows, full-stack user experiences, cloud-native services, and enterprise integrations. You will play a critical role in establishing the evaluation, testing, and monitoring frameworks required to ensure the accuracy, reliability, security, and continuous improvement of AI-powered functionality. \n The ideal candidate combines strong full-stack engineering expertise with hands-on experience developing production-grade AI applications, particularly agentic workflows, retrieval-augmented generation (RAG), and AI evaluation frameworks. \n Key Responsibilities \n Agentic AI Application Development \n\n • Design, develop, and deploy production-grade agentic AI applications using large language models (LLMs), tool calling, structured outputs, and multi-step reasoning workflows.\n, • Build intelligent workflows that interpret user intent, maintain conversation context, orchestrate tools and APIs, and support human review when appropriate.\n, • Implement prompt engineering, context management, workflow orchestration, and error-handling strategies to improve application reliability and response quality.\n, • Integrate retrieval-augmented generation (RAG) capabilities to ground AI-generated responses in trusted enterprise data and source documents.\n\n Full Stack & Cloud-Native Development \n\n • Develop responsive, user-friendly web applications using Angular and modern front-end development practices.\n, • Design and implement scalable backend services and RESTful APIs using C# and .NET.\n, • Develop Python-based AI services and agent orchestration components.\n, • Integrate application services with existing enterprise reporting platforms, databases, authentication providers, and data-access systems.\n, • Build and deploy cloud-native solutions using Microsoft Azure and associated cloud services.\n, • Implement secure API integrations, asynchronous processing, and resilient service-to-service communication.\n\n AI Evaluation, Testing & Quality Engineering \n\n • Own the design and implementation of comprehensive evaluation frameworks for AI-powered applications.\n, • Collaborate with healthcare domain experts and product owners to curate representative evaluation datasets and define scoring rubrics, quality metrics, and acceptance criteria.\n, • Develop automated evaluation processes to assess:\n, • Accuracy and relevance of AI-generated answers.\n, • Correctness and traceability of supporting evidence.\n, • Tool selection, execution, and workflow orchestration.\n, • Context retention and multi-turn conversation quality.\n, • End-to-end task completion and user experience.\n, • Handling of ambiguous, unsupported, and out-of-scope requests.\n, • Combine automated evaluation with human review to validate analytical accuracy and domain-specific correctness.\n, • Establish regression testing and baseline comparisons to identify quality degradation resulting from changes to models, prompts, datasets, or application code.\n, • Integrate AI evaluation suites, automated tests, and quality gates into CI/CD pipelines.\n, • Maintain version-controlled prompts, evaluation datasets, test configurations, and performance baselines.\n\n Reliability, Security & Production Operations \n\n • Design resilient execution workflows incorporating timeouts, retries, cancellation, durable state management, and recovery mechanisms.\n, • Implement authentication, authorization, and data-access controls across APIs, AI agents, tools, and underlying data sources.\n, • Ensure sensitive healthcare and enterprise data is handled securely and in accordance with applicable organizational policies.\n, • Establish observability through application logging, distributed tracing, performance monitoring, and error reporting.\n, • Monitor production AI quality, response latency, operational costs, and failure rates.\n, • Analyze production feedback and failures, convert findings into reproducible test cases, and drive measurable improvements.\n, • Optimize application performance, scalability, reliability, and cloud operating costs.\n\n Data Engineering & Analytical Integrity \n\n • Develop and optimize SQL queries, relational data models, and data-access services.\n, • Ensure analytical results are accurate, reproducible, and supported by traceable source data.\n, • Address data-quality challenges, including missing values, inconsistencies, numerical precision, and aggregation accuracy.\n, • Work closely with business stakeholders and healthcare domain experts to validate analytical findings and communicate uncertainty and system limitations.\n\n Required Qualifications \n\n • 8+ years of professional software engineering experience delivering and supporting secure, scalable enterprise applications.\n, • Hands-on experience designing and developing agentic AI applications using LLMs, tool calling, structured outputs, prompt engineering, and multi-step workflows.\n, • Strong full-stack development experience with Angular, C#/.NET Web APIs, and Python.\n, • Practical experience building AI evaluation and quality pipelines, including evaluation datasets, scoring methodologies, automated regression testing, and human-in-the-loop validation.\n, • Demonstrated ability to use evaluation results and production feedback to improve AI application quality and reliability.\n, • Strong experience with SQL, relational database design, data modeling, and analytical data processing.\n, • Experience building and deploying cloud-native applications using Microsoft Azure.\n, • Hands-on experience with CI/CD pipelines, automated unit testing, integration testing, and end-to-end testing.\n, • Strong understanding of enterprise application security, authentication, authorization, secure API design, and data-access controls.\n, • Experience implementing observability, distributed tracing, asynchronous processing, and fault-tolerant application architectures.\n, • Strong analytical, problem-solving, communication, and collaboration skills.\n, • Ability to work independently and effectively collaborate with engineering teams, product owners, and business/domain experts.\n\n Preferred Qualifications \n\n • Experience implementing retrieval-augmented generation (RAG), including document ingestion, embedding generation, vector search, retrieval strategies, and grounding evaluation.\n, • Experience with healthcare analytics, enterprise reporting platforms, or data-intensive applications involving sensitive information.\n, • Experience transitioning AI proof-of-concept solutions into production-grade enterprise applications.\n, • Familiarity with AI quality metrics, LLM-as-a-judge approaches, semantic evaluation, groundedness assessment, and human evaluation workflows.\n, • Experience optimizing AI application performance, model selection, inference latency, token consumption, and cloud operating costs.\n, • Familiarity with Azure AI services and enterprise AI application architecture.\n\n \n