Senior Databricks Solution Architect
hace 4 días
Norwich
Senior Databricks Solution Architect Unison Group Norwich, England, GB We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using the Databricks Lakehouse Platform. The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions. • Design and architect Databricks Lakehouse solutions for large-scale enterprise data platforms., • Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization., • Define Databricks workspace architecture, cluster strategies, security models, and operational standards., • Design and develop scalable data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake., • Implement data ingestion frameworks supporting batch and real-time data processing., • Design Delta Lake architectures including:, • Medallion architecture (Bronze/Silver/Gold layers), • Data quality frameworks, • Schema evolution and optimization, • Perform Databricks performance tuning:, • Spark job optimization, • Query tuning, • Implement Databricks governance and security using:, • Unity Catalog, • Access controls, • Audit capabilities, • Lead migration projects from legacy data platforms to Databricks Lakehouse., • Define best practices for Databricks CI/CD, DevOps, and environment management., • Collaborate with data engineers, data scientists, analysts, and enterprise architects., • Provide technical guidance, architecture documentation, and design reviews., • Strong hands-on experience with Databricks Platform., • Expert knowledge of:, • PySpark / Scala, • SQL, • Data Lakehouse architecture, • Experience designing and delivering enterprise Databricks solutions., • Strong experience with cloud platforms:, • AWS Databricks, • Experience with:, • Databricks Workflows, • Jobs orchestration, • Notebooks, • Repos, • Cluster management, • Unity Catalog, • Databricks Certified Data Engineer Professional / Databricks Certified Architect., • Experience with:, • MLflow, • Feature Store, • MLOps, • Knowledge of streaming technologies:, • Structured Streaming, • Kafka, • Experience with data governance, compliance, and enterprise security frameworks., • Strong Databricks architecture and implementation expertise., • Ability to design scalable and cost-efficient Lakehouse solutions., • Strong troubleshooting and performance optimization skills., • Ability to lead technical discussions with enterprise stakeholders. Full-time #J-18808-Ljbffr