Data Engineer

Overview: The Data Engineer is responsible for developing, maintaining, and supporting data pipelines using Python and SQL Server (T-SQL), while assisting with data integration from internal and external systems within a modern Azure-based data platform. The role supports critical data operations in regulated environments and provides opportunities for hands-on learning and growth under the guidance of senior data engineers. Job Description: • Develop and maintain ETL pipelines using Python and SQL Server for data ingestion, transformation, and integration from various data sources, including structured files and RESTful APIs. • Write, test, and debug Python code for data processing, automation, and basic integrations following established standards and best practices. • Create and maintain T-SQL queries, views, and stored procedures to support business logic and reporting requirements. • Assist in building and operating data workflows using Azure Data Factory, Azure SQL, and Azure Blob Storage. • Support the monitoring of data pipelines and help troubleshoot data quality issues, pipeline failures, and performance problems. • Follow defined data quality, security, and compliance procedures in regulated environments such as healthcare and financial services. • Collaborate with data analysts, software engineers, and DevOps teams to understand data requirements and upstream systems. • Participate in code reviews, implement feedback, and continuously improve coding and engineering practices. • Create and maintain clear documentation for data pipelines, logic, and operational processes. Qualifications and Experience: Education • Bachelor’s Degree — Preferred Experience • 1–4 years of professional experience in Data Engineering, Software Engineering, or a related technical role — Required • Strong hands-on proficiency in Python, including writing functions, handling errors, and debugging code — Required • Experience working with relational databases, preferably SQL Server — Required • Strong working knowledge of SQL, including joins, aggregations, subqueries, and basic performance considerations — Required • Exposure to integrating or consuming data from RESTful APIs or external data sources — Required • Familiarity with Git or similar version control systems — Required • Strong analytical and problem-solving skills with the ability to learn quickly — Required Preferred Experience • Experience with additional Azure services such as Azure Synapse, Azure DevOps, and Azure Functions • Understanding of data modeling and warehousing concepts • Exposure to cybersecurity data, SIEM tools, or SOC operations • Knowledge of .NET Framework and C#-based APIs, particularly in data consumption contexts • Background in MSP/MSSP environments or consulting • Familiarity with Power BI or other data visualization tools Knowledge, Skills, and Abilities: • Exposure to Azure data services such as Azure Data Factory, Azure SQL, or Blob Storage. • Basic understanding of ETL concepts, data validation, and pipeline monitoring. • Familiarity with data modeling fundamentals, including tables, keys, and relationships. • Awareness of data security, privacy, and compliance standards such as HIPAA and SOC2. • Experience with Power BI or other data visualization tools. • Basic understanding of application systems or APIs built using .NET or similar frameworks. • Ability to design and support scalable ETL pipelines using Python and SQL Server for data ingestion, transformation, and integration from diverse sources. • Ability to develop and optimize T-SQL stored procedures to support business logic and reporting needs. • Ability to support secure and efficient data workflows using Azure Data Factory, Azure SQL, and Azure Functions. • Ability to help ensure data quality, lineage, and compliance requirements are maintained. • Ability to collaborate with software engineering, data analytics, security, and DevOps teams. • Ability to monitor and troubleshoot pipeline failures and data discrepancies. • Ability to participate in code reviews and continuously improve engineering practices. Attributes that will drive success: • Ability to work effectively in a collaborative, fast-paced environment. • Good written and verbal communication skills.