Head of AI Engineering - Legal Innovation & Automation
1 day ago
Melville
Job Description RESUME SUBMISSION REQUIREMENTS: We are seeking a highly skilled Artificial Intelligence Programmer to lead our AI development team. In addition to submitting your resume, candidates must provide a concise summary (no longer than one side of a page) highlighting their experience in leading teams focused on our key short term AI initiatives. These include document summarization with hypertext links to source documents, real-time phone call feedback systems, the creation of generative AI deliverables (e.g., legal demand letters), agentic workflows, and ad hoc queries to databases. Experience in writing comprehensive architecture documents, epics, and acceptance criteria is also required. This role demands a visionary leader with proven expertise in AI-driven projects and team management. About SmartAdvocate SmartAdvocate is the award-winning, enterprise-class legal case-management platform trusted by hundreds of law firms. We are launching a multi-year program to weave Large Language Models, real-time speech analytics, predictive insight, and autonomous agents into every step of a legal matter. We seek a hands-on AI Architect / AI Tech Lead / Director of AI who can lead a team of AI developers to turn ambitious ideas into secure, scalable, production systems. What You’ll Do • Own the AI Architecture – Design the end-to-end stack (RAG pipelines, vector stores, GPU inference clusters, event-driven micro-services, real-time audio services, and agent orchestration—all hardened for HIPAA, SOC 2 and privileged work-product., • Set Technical Direction – Evaluate GPT-4 / Claude / Gemini vs. open-weights (Llama 3, Mistral, Claude 3 Opus, etc.). Build fine-tuning and RAG pipelines with LangChain, LlamaIndex, CrewAI, AutoGen, MetaGPT, and deploy via vLLM/TGI on-prem or VPC GPUs. Establish coding and DevOps standards for the AI team., • Lead/Director Delivery – Break down a 12-month roadmap into iterative releases (document summarization with hypertext links to source documents, live phone call feedback, generative AI deliverable including legal demand letters, agentic workflows, ad hoc queries to database), writing architecture docs, epics and feature acceptance criteria., • Hands-on Prototyping & Code Reviews – Build reference implementations in Python/C#, mentor engineers, and enforce best practices for prompt engineering, evaluation harnesses, and CI/CD., • Security & Compliance Champion – Implement HIPAA-ready de-identification, encryption, audit logging, and model-governance controls; draft architecture for BAAs and SOC 2 evidence., • Cross-Functional Collaboration – Work with the CTO, product managers, and legal SMEs to align AI capabilities with user value, timelines, and budget., • Performance & Cost Optimization – Right-size GPU resources, tune model latency, and refine retrieval techniques to deliver sub-second answers where needed., • Talent Development – Guide a small team of 4-6 ML and backend engineers, fostering a culture of experimentation and high-quality engineering., • Agentic & Real-Time Systems - Lead development of agentic AI that suggests and (with approval) executes multi-step workflows, plus live call / Zoom analysis that delivers sub-second feedback, sentiment scores, and action-item extraction., • 10+ yrs enterprise software; 5+ yrs ML/AI; 2+ yrs production LLM or generative-AI deployments., • Strong written and verbal English communication skills are essential, as this role involves collaborating with cross-functional U.S.-based teams, writing technical documentation, and participating in product planning meetings., • Proven track record architecting self-hosted LLM systems (Llama 2/3, Mistral, Claude, etc.) with fine-tuning, Retrieval-Augmented Generation, and vector databases (Pinecone, Weaviate, Qdrant, Elasticsearch vector search)., • Hands-on expertise with real-time speech-to-text (OpenAI Whisper, Azure/GCP Speech, Deepgram, or similar), real-time NLP analytics, and WebRTC/Socket.io streaming., • Fluency in LLM frameworks (LangChain, LlamaIndex) and agent orchestration; comfortable implementing function-calling and workflow agents. Production experience building agent-orchestrated workflows using frameworks such as CrewAI, AutoGen, or MetaGPT., • Deep Python (FastAPI, asyncio) and C#/.NET skills; comfortable reviewing PRs in both., • Familiarity with legal terminology, litigation lifecycles, and HIPAA-compliant healthcare data handling., • Kubernetes, Docker, and GPU orchestration (NVIDIA Triton, K8s GPU operators); strong DevSecOps mindset., • API design & integration: REST/GraphQL, event buses (Kafka, NATS), webhook patterns., • Demonstrated success building intelligent chatbots and agentic AI systems that automate workflows, enhance client engagement, and deliver measurable productivity gains., • Prior work in legal-tech or other regulated domains (finance, healthcare), • MS SQL Server tuning; ASP.NET MVC/Web API., • Experience with PACER, Westlaw, LexisNexis, or contract-analysis tools (Harvey, Spellbook)., • Exposure to monitoring stacks (Langfuse, MLflow, Prometheus/Grafana) and differential privacy/federated learning., • High Impact + Autonomy – Architect the AI backbone that will redefine legal case management., • Modern Stack, Real Budgets – State-of-the-art GPUs, freedom to choose the best open-source and commercial tech., • Inclusive Culture – Direct access to the CTO, rapid decision cycles, and a collaborative, growth mindset., • Competitive Package – Excellent salary + bonus, medical/dental/vision, 401(k), generous PTO, flexible hybrid schedule. Apply on Indeed with your résumé and a one-page case study describing a production AI system you architected—highlight its scale, security controls, and measurable business impact. Job Type: Full-time Pay: $150,000.00 - $200,000.00 per year Benefits: • 401(k), • 401(k) matching, • Dental insurance, • Flexible spending account, • Health insurance, • Health savings account, • Life insurance, • Paid time off, • Parental leave, • Are you able to work in our Melville, NY office at least 50% of your time? Company DescriptionAward-Winning Legal Case Management Software Featuring Built-in Artificial Intelligence.Award-Winning Legal Case Management Software Featuring Built-in Artificial Intelligence.