Data Platform Engineer officer
State Street
Data Platform Engineer officer
Job Description
Role Summary Design, build, and run core services and reusable components for the open-architecture data platform, enabling scalable ingestion, metadata-driven processing, governance, and reliable multi-engine data consumption. Partner with Platform Engineering and Data Engineering teams to provide secure, observable, production-grade capabilities that support Core Reference, IBOR, and Stamford data initiatives. Key Responsibilities Design and build backend services and shared libraries that provide platform capabilities, including: Streaming & batch ingestion frameworks Metadata-driven pipelines API and consumption layer integrations Transformation engine and MDM engine Engineer reusable platform components aligned with open formats (Iceberg, Parquet), catalog systems (Polaris, Horizon, Unity Catalog), and query engines (Spark, Trino) Implement platform security controls including role-based access control, authentication, and integration with enterprise entitlements Build and maintain platform observability (logging, metrics, traces), alerting, and operational dashboards in support of SLAs Collaborate with architects to define scalable, extensible designs and ensure adherence to open standards Drive engineering excellence through code reviews, automated testing, CI/CD, and production readiness practices Ensure platform services meet enterprise security and compliance requirements (SCC, governance) Participate in on-call / major incident management and troubleshoot production issues to maintain platform stability Qualifications Strong programming skills in Python and Java; knowledge of Rust is a bonus Strong programming skills in Spark, with hands-on knowledge of optimization and debugging Hands-on experience with Snowflake and Databricks as data platforms Strong understanding of SQL optimization, execution, and large-scale analytics Deep understanding and working knowledge of open table formats such as Apache Iceberg are preferred Deep understanding of distributed computing and hyperscale Experience building production-grade services in cloud environments; AWS,GCP and/or Azure is preferred Familiarity with data catalogs (any of Polaris, Horizon, Unity) and metadata management Familiarity with streaming pipelines and Kafka is preferred Understanding of data modeling and data product design Experience with CI/CD, containers, and Kubernetes Financial/data platform background preferred but not required About State StreetAcross the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.Discover more information on jobs at StateStreet.com/careersRead our CEO Statement
Locations
- China, Hangzhou
Skills Required
- Rust is a bonusintermediate
- optimizationintermediate
- Snowflakeintermediate
- open table formats such as Apache Iceberg are preferredintermediate
- production-grade services in cloud environmentsintermediate
- data catalogsintermediate
- streaming pipelinesintermediate
- CI/CDintermediate
Required Qualifications
- Strong programming skills in Python and Java; knowledge of Rust is a bonus (experience)
- Strong programming skills in Python and Java; knowledge of Rust is a bonus (experience)
- Strong programming skills in Spark, with hands-on knowledge of optimization and debugging (experience)
- Strong programming skills in Spark, with hands-on knowledge of optimization and debugging (experience)
- Hands-on experience with Snowflake and Databricks as data platforms (experience)
- Hands-on experience with Snowflake and Databricks as data platforms (experience)
- Strong understanding of SQL optimization, execution, and large-scale analytics (experience)
- Strong understanding of SQL optimization, execution, and large-scale analytics (experience)
- Deep understanding and working knowledge of open table formats such as Apache Iceberg are preferred (experience)
- Deep understanding and working knowledge of open table formats such as Apache Iceberg are preferred (experience)
- Deep understanding of distributed computing and hyperscale (experience)
- Deep understanding of distributed computing and hyperscale (experience)
- Experience building production-grade services in cloud environments; AWS,GCP and/or Azure is preferred (experience)
- Experience building production-grade services in cloud environments; AWS,GCP and/or Azure is preferred (experience)
- Familiarity with data catalogs (any of Polaris, Horizon, Unity) and metadata management (experience)
- Familiarity with data catalogs (any of Polaris, Horizon, Unity) and metadata management (experience)
- Familiarity with streaming pipelines and Kafka is preferred (experience)
Preferred Qualifications
- Understanding of data modeling and data product design (experience)
- Understanding of data modeling and data product design (experience)
- Experience with CI/CD, containers, and Kubernetes (experience)
- Experience with CI/CD, containers, and Kubernetes (experience)
- Financial/data platform background preferred but not required (experience)
- Financial/data platform background preferred but not required (experience)
Responsibilities
- Design and build backend services and shared libraries that provide platform capabilities, including:
- Design and build backend services and shared libraries that provide platform capabilities, including:
- Streaming & batch ingestion frameworks
- Streaming & batch ingestion frameworks
- Metadata-driven pipelines
- Metadata-driven pipelines
- API and consumption layer integrations
- API and consumption layer integrations
- Transformation engine and MDM engine
- Transformation engine and MDM engine
- Engineer reusable platform components aligned with open formats (Iceberg, Parquet), catalog systems (Polaris, Horizon, Unity Catalog), and query engines (Spark, Trino)
- Engineer reusable platform components aligned with open formats (Iceberg, Parquet), catalog systems (Polaris, Horizon, Unity Catalog), and query engines (Spark, Trino)
- Implement platform security controls including role-based access control, authentication, and integration with enterprise entitlements
- Implement platform security controls including role-based access control, authentication, and integration with enterprise entitlements
- Build and maintain platform observability (logging, metrics, traces), alerting, and operational dashboards in support of SLAs
- Build and maintain platform observability (logging, metrics, traces), alerting, and operational dashboards in support of SLAs
- Collaborate with architects to define scalable, extensible designs and ensure adherence to open standards
- Collaborate with architects to define scalable, extensible designs and ensure adherence to open standards
- Drive engineering excellence through code reviews, automated testing, CI/CD, and production readiness practices
- Drive engineering excellence through code reviews, automated testing, CI/CD, and production readiness practices
- Ensure platform services meet enterprise security and compliance requirements (SCC, governance)
- Ensure platform services meet enterprise security and compliance requirements (SCC, governance)
- Participate in on-call / major incident management and troubleshoot production issues to maintain platform stability
- Participate in on-call / major incident management and troubleshoot production issues to maintain platform stability
Tags & Categories
Answer 10 quick questions to check your fit for Data Platform Engineer officer @ State Street.
10 Questions
~2 Minutes
Instant Score