Senior Applied Deep Learning Research Scientist, Multi-Modal LLMs

Nvidia - Santa Clara, CA

Full-time - Senior

Santa Clara, CA

Computer and Electronic Product Manufacturing

About the position

NVIDIA is seeking Senior Applied Deep Learning Research Scientists to join our innovative applied deep learning research team, which has been at the forefront of advancements in multi-modal large generative models. Our team has pioneered significant technologies such as Megatron, DLSS, and various audio models including BigVGAN, One TTS Aligner To Rule Them All, RADMM, ZenFlow, and P-Flow. We are dedicated to pushing the boundaries of "Generative AI" by developing state-of-the-art multi-modal generative models that address real-world challenges. If you are enthusiastic about the latest research and technologies that are revolutionizing "Generative AI" and are eager to explore creative new paradigms for multi-modal generative models, particularly focusing on large language models (LLMs) and audio, this position is an excellent opportunity for you. In this role, you will be responsible for researching, designing, implementing, and publishing novel, large-scale generative models that enhance multi-modal LLMs with a specific emphasis on audio. You will design and implement machine learning techniques to adapt generative models for downstream tasks such as audio understanding and synthesis. Additionally, you will construct and curate datasets for large-scale machine learning applications across various domains. Collaboration is key; you will work closely with other team members, research teams, and product teams to integrate your research and developments into industry-leading products. This position offers a unique chance to contribute to groundbreaking research while having a tangible impact on real-world applications.

Responsibilities

• Research, design, implement and publish novel, large-scale generative models that improve multi-modal LLMs with a focus on audio.
• Design and implement machine learning techniques to adapt generative models to downstream tasks of interest such as audio understanding and synthesis.
• Construct and curate datasets for large-scale machine learning and specific domains of applications.
• Collaborate with other team members, research teams, as well as product teams to integrate your research and developments into products.

Requirements

• Must hold a Masters Degree or PhD in Computer Science/Engineering, Statistics, Electrical Engineering, or equivalent experience.
• Excellent knowledge of the theory and practice of "generative AI".
• 5+ years of relevant research experience, with at least 3+ years of industry experience, in deep learning, audio, generative models, or related machine learning fields.
• Strong knowledge of application areas such as audio and natural language processing.
• Excellent programming skills in rapid prototyping environments such as Python; C++ and parallel programming (e.g., CUDA) is a plus.
• Expertise with deep learning frameworks such as PyTorch.
• Outstanding research track record.
• Excellent communication skills.

Benefits

• Competitive salaries
• Generous benefits package
• Equity options