Senior Databricks Administrator/Architect - valència
hace 5 días
Valencia
Overview In this remote role you guide the design and delivery of enterprise data platform solutions on Azure Databricks for a major client, within the Development team. You help shape modern Lakehouse architectures and governance, collaborating with cross‑functional groups to scale data workloads and ensure secure, automated deployments. You’ll optimize Spark workloads, implement CI/CD for Databricks, and support migration from legacy data platforms. This role offers impact across governance, security, and real‑world analytics at scale in an international environment. You will join a mission‑driven, enterprise‑grade IT services provider delivering high‑value data solutions. Compensaciones / Beneficios • competitive remuneration, • remote work option, • full benefits package Responsabilidades, • Collaborate with Data Platforms and Integration teams to architect and implement scalable enterprise data platform solutions using Azure Databricks and related Azure services, • Design and develop data engineering frameworks for batch, streaming, ELT/ETL, and real-time analytics workloads, • Administer Azure Databricks workspaces: cluster config, job scheduling, Unity Catalog, secret management, and workspace governance, • Define and implement enterprise security architecture: Azure AD integration, RBAC, PIM, encryption, private networking, Key Vault, • Establish data governance standards with Unity Catalog: data lineage, metadata management, access policies, audits, • Deploy and configure CI/CD pipelines with Azure DevOps and DAB; IaC with Terraform/Bicep; automate deployments and environment promotion, • Optimize Databricks workloads: performance tuning, Spark optimization, partitioning, Delta optimization, Photon, cost control, • Design monitoring and operational dashboards using Azure Monitor, Log Analytics, Application Insights and Databricks system tables, • Develop reusable architecture patterns, development standards, naming conventions, and platform documentation, • Support migration of legacy data warehouses and ETL solutions to Azure Databricks and Lakehouse architectures, • Provide technical support and troubleshooting for production incidents related to Azure Databricks, Spark workloads, data pipelines, and platform infra Requisitos principales, • University degree with relevant IT experience (9+ years), • 8+ years designing and implementing enterprise data platforms, • 5+ years hands-on Azure Databricks in enterprise environments, • Experience with Azure Databricks, Delta Lake, Unity Catalog, and Databricks governance, • Experience with Apache Spark (PySpark and/or Spark SQL), • Experience with Terraform and/or Bicep for IaC, • Strong CI/CD experience with Azure DevOps and deployment automation, • Security architecture experience: RBAC, Managed Identities, Private Endpoints, Key Vault, encryption, • Data governance, metadata, lineage, and fine-grained access controls, • Strong SQL, Python and PySpark skills, • Excellent command of English, • strong communication, • collaborative mindset, • problem-solving, • Azure Databricks, • Delta Lake, • Unity Catalog
Jose Vicente R.Activo hace 1 hora