MSYS - Redmond, WA
posted 2 months ago
The Data Engineer position is a long-term onsite role located in Redmond, WA, specifically targeting local candidates. The ideal candidate will have a minimum of 10 years of experience, with a strong emphasis on Microsoft technologies. The last 3 to 4 years of experience should include proficiency in using Pyspark and working with Kusto for telemetry data. The role requires the ability to write Python code in Azure Synapse Notebook and a solid understanding of Azure Synapse from end to end. Candidates should also have experience handling large volumes of data, potentially in the hundreds of gigabytes, and be capable of debugging and optimizing jobs in notebooks for efficient memory consumption and processing. In addition to technical skills, the candidate should possess knowledge of data warehousing and data management practices to ensure efficient data handling. Familiarity with data modeling and data engineering tasks is also essential. Experience with large datasets using SQL, Azure Data Lake, PySpark, ADF, Synapse, and similar technologies to derive actionable insights is highly desirable. The candidate is expected to be productive soon after joining and should have a good understanding of the Microsoft ecosystem, including Azure DevOps, Azure WebApps, Azure Functions, Azure Cosmos DB, and Event Hubs. The Data Engineer will be responsible for providing technical guidance and solutions, advocating for and implementing best practices and coding standards within the team. They will also develop and mentor team members to enhance their technical capabilities and increase overall productivity. Ensuring process compliance in the assigned module is crucial, as is participation in technical discussions and reviews as a consultant for feasibility studies, technical alternatives, best packages, supporting architecture best practices, technical risks, and component breakdowns for estimations. Additionally, the Data Engineer will prepare and submit status reports to minimize exposure and risks on the project and manage the closure of escalations.