Senior Data Engineer | PySpark & Fabric

Tecknoworks is a global technology consulting company. At our core, we embody values that define who we are and how we operate. We are curious, continuously seeking to expand our understanding and question conventional wisdom. Fearlessness drives us, propelling us to take daring steps to achieve significant outcomes. Our aspiration to be inspiring motivates us to consistently reach for our personal and collective best, setting an example for ourselves and those we interact with. Collaboration is our strength, capitalizing on the diverse brilliance within our team. Our commitment knows no bounds; we consistently go the extra mile to ensure enduring positive effects for our clients.

We are seeking a Senior Data Engineer with a passion for clean, efficient, and scalable code. This role is ideal for someone who thrives in a dynamic environment, excels at solving complex problems, and is eager to be a key contributor during a critical technical migration phase. You will play a pivotal role in our transition from SQL to PySpark and Microsoft Fabric while maintaining and improving the current systems.

Key Responsibilities

• Contribute to migrating a large data set to high-quality using Microsoft Fabric.
• Understand the SQL code and translate the code into other SQL dialects using PySpark.
• The system is a big data flow with interdependent calculations hosted in SQL Server and Azure VM services.
• Strive for optimal Azure resource usage with the new solution.
• Write clean, optimal, and well-documented code that aligns with best practices and business logic.
• Mentor junior developers and conduct code reviews.
• Use AI-assisted development tools such as GitHub Copilot, Cursor, or similar to improve development speed and accuracy.
• Collaborate cross-functionally with business stakeholders, analysts, and QA to understand the business context and ensure alignment of technical solutions.
• Proactively identify areas of improvement and suggest scalable solutions. Diagnose and fix bugs in a timely and efficient manner.
• Communicate clearly and effectively within the engineering team and across departments.
• Learn the existing system's business / data domain and optimize the new solution for understandability by domain experts. Organize and name MS Fabric artefacts to be suitable for knowledge sharing between domain experts.

Qualifications

Nice to Have