London
AWS DATA ENGINEER: Domain: Telecommunications Location: London,UK Duration: 12 Months Fixed Term Contract Hybrid working 2-3 Days from Office Keywords: End‑to‑End Data Platform Design, Data Strategy, AWS Cloud Architecture, High‑Availability Architecture, Scalable Data Pipelines ,ETL Architecture, Metadata Management, Iceberg, Batch & Streaming Pipelines, Python, Pypark, Data Pipeline Design & Deployment Role Overview We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain. The candidate will work closely with data architects, business analysts, and cross-functional teams to build scalable and efficient data solutions supporting network analytics, customer insights, billing systems, and telecom OSS/BSS workflows. Key Responsibilities 1. Data Engineering & ETL Development • Design, develop, and maintain ETL/ELT pipelines using AWS-native services (Glue, Lambda, EMR, Step Functions)., • Implement data ingestion from telecom systems like OSS/BSS, CDRs, mediation systems, CRM, billing, network logs., • Optimize ETL workflows for large-scale telecom datasets (high volume, high velocity). 2. Data Warehousing (Redshift) • Build and manage scalable Amazon Redshift clusters for reporting and analytics., • Create and optimize schemas, tables, distribution keys, sort keys, and workload management., • Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg • Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations., • Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark., • Ensure high performance for petabyte-scale telecom datasets (CDRs, tower logs, subscriber activity). 4. Querying & Analytics (Athena) • Develop and optimize Athena queries for operational and analytical reporting., • Integrate Athena with S3/Iceberg for low-cost, serverless analytics., • Manage Glue Data Catalog integrations and table schema management. 5. Storage (S3) & Data Lake Architecture • Design secure, cost-efficient S3 data lake structures (bronze/silver/gold zones)., • Implement data lifecycle policies, versioning, and partitioning strategies., • Ensure data governance, metadata quality, and security (IAM, Lake Formation). 6. Telecom Domain Expertise • Understand telecom-specific datasets such as:, • CDR, xDR, subscriber data, • Network KPIs (4G/5G tower logs), • Customer lifecycle & churn data, • Billing & revenue assurance, • Build models and pipelines to support network analytics, customer 360, churn prediction, fraud detection, etc. 7. Performance Optimization & Monitoring • Tune Spark/Glue jobs for performance and cost., • Monitor Redshift/Athena/S3 efficiency and implement best practices., • Perform data quality checks and validation across pipelines. 8. DevOps & CI/CD (Preferred) • Use Git, CodePipeline, Terraform/CloudFormation for infrastructure and deployments., • Automate pipeline deployment and monitoring. Required Skills • 8–10 years’ experience in data engineering., • Strong hands-on experience with:, • AWS S3, Athena, Glue, Redshift, EMR/Spark, • Apache Iceberg, • Python/SQL, • Experience in telecom data pipelines and handling large-scale structured/semi-structured data., • Strong problem-solving, optimization, and debugging skills. Good to Have Skills • Knowledge of AWS Lake Formation, Kafka/Kinesis, Airflow, or Delta/Apache Hudi., • Experience with ML workflows in telecom (churn, network prediction)., • Exposure to 5G network data models.