Sr. Data Engineer, Data Engineering

About the position

Tenstorrent is leading the industry on cutting-edge AI technology, revolutionizing performance expectations, ease of use, and cost efficiency. With AI redefining the computing paradigm, solutions must evolve to unify innovations in software models, compilers, platforms, networking, and semiconductors. Our diverse team of technologists have developed a high performance RISC-V CPU from scratch, and share a passion for AI and a deep desire to build the best AI platform possible. We value collaboration, curiosity, and a commitment to solving hard problems. We are growing our team and looking for contributors of all seniorities. The Senior Data Engineer in the Data Science and Engineering team will play a key role in fostering a data-driven culture within the organization. This will be achieved by designing, implementing, and maintaining robust data pipelines and architectures that enable efficient and reliable collection, processing, and storage of data from diverse sources. The Senior Data Engineer will guide and empower stakeholders across the organization to create and leverage high-quality, curated datasets and information, thereby facilitating data-driven decision-making and innovation. The ideal candidate will demonstrate proficiency in clarifying ambiguous information, establishing structured data models from unstructured data, and building end-to-end data processing automations running on a modern and scalable hybrid-cloud data infrastructure. Additionally, the candidate should have extensive experience writing Python data processing code, Airflow DAGs, and working with SQL and no-SQL databases. They should view self-serve data analytics and ML applications that leverage the curated datasets they produce as their customers. The ideal candidate highly values and is familiar with IaC, CI, monitoring, and documentation to achieve the highest levels of automation, quality, and functionality. This role is hybrid, based out of Santa Clara, CA. We welcome candidates at various experience levels for this role. During the interview process, candidates will be assessed for the appropriate level, and offers will align with that level, which may differ from the one in this posting.

Responsibilities

• Develop and implement scalable end-to-end data pipelines using modern data processing and orchestration techniques, for both batch and streaming data.
• Lead collaboration with data producers and data consumers to identify clear product requirements and structure an effective implementation of solutions, guiding the overhaul of pre-existing methods, when necessary.
• Identify, evolve, and champion data/software engineering best practices and AI automations, as they best fit a small and agile team.
• Automate infrastructure as code using Terraform.
• Collaborate cross-functionally to define priorities, identify new projects, and establish effective partnerships to contribute to our data engineering roadmap.
• Help envision and build the future of our data platform and data pipelines.

Requirements

• Advanced OOP Python programming skills, particularly in data processing.
• Experience with the SQLAlchemy ORM.
• SQL database and schema design, management, and optimization expertise.
• Hands-on experience with AWS services including S3, EC2, IAM, SFTP, Cloudwatch, and Lambda.
• Familiarity with IaC technologies, such as Terraform.
• Experience with time-series (TimescaleDB) and NoSQL databases (Elasticsearch).
• Experience deploying and working with data analytics, dashboarding and visualization tools, such as Apache Superset, Grafana, Kibana.
• Experience building RESTful APIs.
• Proficiency in containerization technologies (Docker, Kubernetes).
• Experience with Agile development methodologies and tools, including Jira and Git, and familiarity with industry-standard software development lifecycle processes.
• Comfortable working independently and self-motivated to learn and excel in a fast-paced, innovative environment.

Benefits

• Highly competitive compensation package
• Equal opportunity employer