Sr. Data Platform Engineer
26 days ago
Troy
Job Description Job Location – Troy, NY (Hybrid - 3 Days Onsite Per Week) Type - Contract (18 Months) Pay Rate – $70.00 - $78.00 Per Hour (W2 Only) The Central Technology and Services team (CTS) provides data analytics services that support game teams and analysts across the organization. As players perform actions in-game, our platform collects telemetry in real-time. The data collected provides insights for guiding game development, marketing, and monetization. The Data Platform Engineer / DBA manages our database and data platform fleet, maintaining secure, optimized, and highly available databases, data warehouses, and lakehouses. Clien’ts stack is evolving: Redshift remains a core platform, and they are actively expanding into Databricks and open table format (Apache Iceberg on S3). The Data Platform Engineer / DBA works closely with engineering and analytics teams to design, implement, and maintain these systems. Responsibilities • Operate Redshift clusters, Databricks workspaces, RDS/Aurora PostgreSQL instances, and supporting AWS infrastructure, • Perform user/security tasks across platforms: Redshift user/group management, Databricks Unity Catalog access controls, IAM Roles/Policies, RDS parameter and access management, • Design and maintain open table format data lakes using Apache Iceberg on S3, including compaction, snapshot management, partition strategies, and schema evolution, • Profile production workloads and develop strategies to run optimized clusters and workspaces with scale and efficiency, • Provide architecture guidance and support to technical leads; help develop code and SQL for data assets, identify performance tuning opportunities, and work with developers to improve production systems, • Contribute to and maintain automation scripts and internal tooling with production-quality code standards (unit tests, peer review, documentation, Git-based workflows), • Develop monitoring and recovery automation to identify and resolve issues to meet our high SLA, • Research new technologies and develop proofs of concept; propose technical improvements and new capabilities Must Have Skills / Requirements • 3+ years of experience with AWS data services: Redshift, RDS/Aurora PostgreSQL, S3, EC2, Athena, Glue, Lake Formation, • 2+ years of experience with Databricks: workspace administration, Unity Catalog, cluster management, Spark SQL, Delta Live Tables or similar, • 2+ years of experience with PostgreSQL or Aurora PostgreSQL: query tuning, replication concepts, pg_stat views, parameter groups, • 2+ years of experience with AWS Security: IAM (Users, Groups, Roles, Policies, Instance Profiles), Redshift/RDS security, S3 bucket policies Nice to Have Skills / Preferred Requirements • PySpark for data pipeline development, • AWS Glue with Iceberg or Delta Lake integration, • Snowflake administration, • Terraform or AWS CDK for infrastructure as code, • DynamoDB experience, • Feature Store Soft Skills: • Ability to read, understand, debug, and extend complex Python codebases; comfort navigating large repositories with multiple modules and abstractions, • Strong analytical and critical thinking skills, • Ability to lead projects, prioritize, and multitask, • Deadline and detail-oriented, • Part of on-call rotation Technology Requirements: • Experience with Apache Iceberg on S3: partition management, compaction, snapshot management, schema evolution, • Proficiency in Python for automation and shared tooling: object-oriented design, writing maintainable and testable code, participating in code review, and contributing to a shared Git-based codebase, • Experience automating database and platform operations using Python, Jenkins, Airflow, Datadog and Git