Databricks Architect
8 days ago
Leeds
Job Title: Databricks Architect Location: UK/Europe - Remote Engagement Type: Contract (Outside IR35) Role Overview We are seeking a Senior Databricks Architect with 15+ years of experience in Data Engineering/Analytics to design and deliver scalable, secure, and cost-efficient data solutions on AWS. This is a leadership role requiring strong hands-on expertise, proactive risk management, and excellent stakeholder communication skills. Key Responsibilities • Architect and implement data solutions using Databricks on AWS, PySpark, and Python, • Lead cloud migration initiatives, including on-prem to AWS migrations, • Drive GenAI-enabled automation for data asset migration, • Develop scalable data models and deliver actionable business insights, • Design and optimize ETL pipelines (Informatica or similar tools), • Build and deploy PySpark applications for large-scale data processing, • Work on LLM pretraining, fine-tuning, augmentation, and optimization, • Ensure high data quality through cleaning, transformation, and validation (Python/SQL), • Automate data workflows to enhance scalability and efficiency, • Maintain clear technical documentation and best practices Required Experience • 15+ years in Data Analyst/Data Engineer/Data Architect roles, • Strong expertise in Databricks on AWS, • Advanced Python (Pandas, Scikit-learn) and SQL skills, • Proven experience in large-scale cloud data migrations, • Hands-on experience with LLMs and GenAI-based automation, • Strong experience designing secure, high-performance data architectures Key Competencies • Proactive risk & issue management, • Strong requirement clarification and stakeholder engagement skills, • Excellent communication – able to translate technical insights into business value, • Critical thinker with strong problem-solving capabilities, • Self-driven, autonomous, and collaborative approach