The Cloud Platform Engineer focuses on building and operating the environment and ensures a reliable, well-governed, high-performance platform. This role also owns infrastructure, access, tooling, observability, and standards across the full stack.
Main Accountabilities
Provisions, configures, and tunes Databricks workspaces, Unity Catalog, and Apache Spark cluster settings (memory, executors, autoscaling) for automated, cost-efficient compute.Defines and enforces Spark usage standards, query patterns, resource quotas, and Delta Table partitioning guidelines to optimize storage and prevent cluster contention.Manages S3 bucket structures, lifecycle policies, access controls, and overall storage cost governance.Administers and tunes Airflow environments (upgrades, plugins, monitoring), enabling automated DAG authoring and workflow management.Owns IAM roles, policies, and least-privilege access management across all platform servi...