The Data Engineer would be responsible for the following :
1)Assisting in designing and implementing data processes, ETL pipelines, and data models.
2)Writing and optimizing SQL queries for data retrieval, manipulation, and analysis. This involves writing efficient and scalable code to process, transform, and clean large volumes of structured and unstructured data.
3) Developing and maintaining Python scripts for data processing and task automation using libraries such as Pandas, Polars, Dask, or Spark.
4)Collaborating with the Lead Data Engineer to enhance data structures and orchestration frameworks.
5)Manage cloud-based data warehousing solutions, focusing on Snowflake.
6)Utilize Docker for containerizing data applications and workflows.
7) Engage in problem-solving and troubleshooting to ensure data integrity and accessibility.
- Stay current with industry trends and best practices, demonstrating a willingness to learn and adapt.
- Collaborate with internal stakeholders such as data scientists and analysts to understand their requirements and provide them with well-structured and optimized data for analysis and modeling purposes.
Preferred Qualifications:
1) 3-5 years of experience in data engineering or a related field, with a proven track record of success in designing and implementing data processes and architectures.
2) Proficient in SQL, with the ability to write basic to intermediate queries.
3) Strong knowledge of Python and experience with relevant data processing libraries.
4) Prior exposure to cloud-based data warehousing; Snowflake experience is a strong plus.
5) Understanding of data concepts, including ETL, data structures, data modeling, and workflow management.
6) Familiarity with containerization technologies, particularly Docker.
7) Strong analytical and problem-solving skills, coupled with a proactive approach to learning.
8) Knowledge of finance or credit markets is advantageous but not required.
