Data Engineer

The New York City Taxi and Limousine Commission (TLC) is the City agency responsible for regulating for-hire transportation in New York City, including taxis, street hail liveries, high-volume for-hire services such as Uber and Lyft, black cars, luxury limousines, livery vehicles, commuter vans, and paratransit vehicles. The TLC licenses about 180,000 drivers, 118,000 vehicles, and 1,000 businesses, which together transport more than a million passengers a day, making TLC the most active for-hire transportation regulatory agency in the world with oversight of a key component of the City’s transportation network. With the introduction of new apps and technologies, TLC is on the front lines of a rapidly changing mobility landscape and our innovative efforts--whether regulating driver pay, ensuring wheelchair accessibility, working to eliminate traffic fatalities, or preventing discriminatory service--often serve as a model for other cities.

We are looking for a motivated Data Engineer to join our Data Analytics Unit within the Policy Division. You'll be working with large datasets, including detailed trip data for every taxi and for-hire vehicle in New York City. The Data Engineer will work closely with other team members to understand the team’s ongoing data ingestion and processes. This role is crucial in processing and analyzing billions of trip records, requiring expertise in big data technologies and distributed computing. The data includes detailed information on passenger wait times, pickup and drop-off locations, minute-level breadcrumb pings, passenger costs, and driver pay.

We’re seeking a Data Engineer who is excited to work with big data to help shape transportation policy in NYC. You will manage our data pipelines, enhance data governance, and bridge the gap between modern cloud-based solutions and legacy systems. Your work will involve building and maintaining the data and the technologies which are tied to our team’s ongoing projects such as traffic congestion, fare regulation, driver pay, vehicle electrification, and charging infrastructure.

As a Data Engineer on our team, you will be performing the following tasks:
- Design, implement, and maintain scalable data pipelines.
- Own internal data validations and documentation processes along with the data engineering team.
- Collaborate with data analysts and policy researchers on a variety of projects, including translating business and policy requirements into data schemas and models.
- Develop and maintain tools for automating and modernizing our data processes.
- Liaise with other departments to bridge gaps in existing processes and tools.
- Track, test, develop and deploy code.
- Analyze existing analytics code, modularizing and refactoring code where needed.
- Troubleshoot and complex data issues and implement robust solutions.

1. For Assignment Level I (only physical, biological and environmental sciences and public health) A master's degree from an accredited college or university with a specialization in an appropriate field of physical, biological or environmental science or in public health.
To be appointed to Assignment Level II and above, candidates must have:
1. A doctorate degree from an accredited college or university with specialization in an appropriate field of physical, biological, environmental or social science and one year of full-time experience in a responsible supervisory, administrative or research capacity in the appropriate field of specialization; or
2. A master's degree from an accredited college or university with specialization in an appropriate field of physical, biological, environmental or social science and three years of responsible full-time research experience in the appropriate field of specialization; or
3. Education and/or experience which is equivalent to "1" or "2" above. However, all candidates must have at least a master's degree in an appropriate field of specialization and at least two years of experience described in "2" above. Two years as a City Research Scientist Level I can be substituted for the experience required in "1" and "2" above.

NOTE:
Probationary Period
Appointments to this position are subject to a minimum probationary period of one year.

The City of New York is an inclusive equal opportunity employer committed to recruiting and retaining a diverse workforce and providing a work environment that is free from discrimination and harassment based upon any legally protected status or protected characteristic, including but not limited to an individual's sex, race, color, ethnicity, national origin, age, religion, disability, sexual orientation, veteran status, gender identity, or pregnancy.