Data Engineer II

We are seeking a skilled Database Engineer to join our team within the Division of Data Driven and Digital Medicine (D3M) and Institute for Critical Care Medicine (ICCM). In this role, you will be responsible for designing, implementing, and maintaining a robust datamarts, data warehouses, databases and data visualization platforms for healthcare data. You will utilize your expertise in database management systems, to work with both current ETL/Data Warehousing and future Big Data/Streaming/Pipeline architectures. The focus will be on choosing optimal solutions to use for these purposes, then implementing, maintaining, and monitoring them, always being mindful of the overarching goal of accelerating translational research and improving clinical care.

 

1. Facilitate data collection from a variety of different sources, getting it in the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly and with a common standard interface.
2. Ensure that data streams/pipelines are scalable, repeatable, and secure, and can serve multiple users. 
3. Design and develop a comprehensive datamart framework and data visualization platforms for healthcare data, ensuring scalability, efficiency, and data integrity.
4. Manage and administer databases on relational database management systems such as Postgres or Microsoft SQL Server, optimizing performance and ensuring high availability.
5. Develop and implement Extract, Transform, Load (ETL) processes to seamlessly integrate healthcare data from sources including Epic Inc's Caboodle and Clarity databases into the data mart.
6. Responsible for creating the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly.
7. Additional responsibilities include developing prototypes and proof of concepts for the selected solutions, and implementing complex big data projects with a focus on collecting, parsing, and managing large sets of data using multiple platforms to allow for Research and Data Science initiatives.
8. Translate business requirements into modern data pipeline solutions. Create centralized documents and diagrams of all solutions.
9. Design and implement monitoring, backup, and disaster recovery of data systems.
10. Create a data catalog store of all metadata.
11. Responsible for the integrity and security of data in all forms of storage throughout the Data Architecture. Ensure compliance with the Institutional Review Board and HIPAA to follow all applicable policies and procedures.
12. Assist in the development of standards and procedures affecting data management, design and maintenance. Documents all standards and procedures.
13. Possess an extremely flexible attitude. Willing to work with multiple types of technologies and languages with an open mind and without technology bias. Continuous interest in updating skill sets and knowledge of trends in the Big Data Technology space.
14. Work closely with cross-functional teams including data scientists, healthcare providers, and IT professionals to understand data requirements, develop solutions, and support data-driven decision-making.
15. Other duties as assigned 

• Bachelor's degree in Computer Science, Information Technology, or a related field. Master's degree preferred.
• Strong proficiency in ETL processes, data warehousing, and data integration techniques with proven experience in database development and management
• Strong SQL and NoSQL database knowledge: Oracle, PostgreSQL/MYSQL, Mongo DB (or similar)
• In-depth knowledge of healthcare data systems and standards, including experience with Epic Inc's Caboodle and Clarity databases is a big plus
• Working knowledge of cloud architecture and implementation on Azure or AWS, is a big plus. Experience with serverless computing, creating VMs, cloud security, and other cloud services is also a big plus.
• Excellent analytical and problem-solving skills with a keen attention to detail.
• Effective communication skills and ability to collaborate in a team environment
• Hybrid (New York City + remote)

Non-Bargaining Unit, 024 - Personalized Medicine Inst - ISM, Icahn School of Medicine