From Software Engineer to AI Engineer – with Janvi Kalra

Episode Overview

• Janvi Kalra shares a mental framework for navigating the complex AI job market by categorizing companies as either Product, Infrastructure, or Model focused.
• She discusses the importance of a "learning by doing" approach in the fast-evolving AI space, emphasizing hands-on experience from hackathons and personal projects over traditional resources.
• The conversation covers her personal journey of interviewing at 46 companies, leading to her role at OpenAI, and the strategies she used to focus her search.
• Janvi provides a look inside OpenAI's unique engineering culture, which combines the rapid iteration of a startup with the challenge of building for massive scale.
• The discussion explores how AI is transforming the role of the software engineer, blurring disciplinary lines while reinforcing the need for fundamental skills like debugging and system design.

Key Concepts

• AI Company Framework: The AI industry can be understood through three distinct categories: Product companies (building applications on models), Infrastructure companies (building tools for AI products), and Model companies (building the foundational AI intelligence).
• Learning by Doing: In a field where formal documentation is often outdated, the most effective way to gain expertise is through hands-on work, such as participating in hackathons and building personal projects.
• Proactive Career Growth: Taking initiative to prototype solutions for user problems can lead to significant career opportunities and product impact, as demonstrated by a hackathon project evolving into a major company initiative.
• High-Growth AI Culture: Companies like OpenAI possess a unique engineering environment characterized by a dual focus on high-speed iteration and building for immense production scale, granting engineers high levels of trust and autonomy.
• The Evolving Software Engineer Role: AI tools are blurring the lines between product management, design, and engineering, empowering engineers to take on broader responsibilities but also demanding deeper systems knowledge for when things go wrong.

Quotes

• At 0:09 - "In terms of the space, there are the product companies, infrastructure companies, and the model companies." - Kalra introduces her three-part framework for understanding and categorizing the AI industry.
• At 1:08 - "I decided to focus on model and infrastructure companies because I wanted to keep getting breadth in what I was doing, and I felt like the product companies were too similar to my experience at Coda." - She details her personal job search strategy, which was driven by a desire for growth and new challenges.
• At 1:21 - "The tradeoff was that it was a bit more of an uphill battle because the work that I had done was not as relevant to model or infrastructure companies." - Kalra acknowledges the difficulty of pivoting from a product role to the more foundational layers of the AI stack.
• At 28:00 - "I found it most helpful to learn by just doing." - Janvi explains that because the AI field changes so quickly and lacks static resources like books, hands-on experimentation at hackathons was her primary learning method.
• At 33:01 - "It became the birth of a second product for Coda, called Coda Brain... from January to June of 2024, we had a much larger initiative where 20 people were now working on it." - Janvi describes how her small, experimental hackathon project to build a workspace Q&A tool quickly evolved into a major, company-wide product initiative.
• At 38:45 - "If it's not 'heck yes,' and if you have savings, don't join. It's not fair to the company or you." - Janvi shares a piece of advice from a mentor that guided her decision-making process during her extensive job search in the volatile AI startup scene.
• At 43:39 - "When shit hits the fan and you're in a sev, AI doesn't help that much because it doesn't work very well in between at a high systems level and then reading logs." - Janvi makes the point that while AI is great for generation, engineers must still cultivate a deep understanding of their systems to handle critical production incidents.
• At 51:06 - "I think what makes OpenAI unique is the mix of speed and building for scale... You just think normally you get one or the other, and it's really fun to get both." - Janvi describes the distinct engineering culture at OpenAI, which combines the fast-paced, iterative environment of a startup with the immense user scale of a large tech company.
• At 53:41 - "My observation is that the systems are built to enable you to ship fast, and they give engineers a lot of trust... When I joined, you could make stat sig changes without an approval... you get to deploy with one review immediately." - Janvi highlights the high degree of autonomy and rapid deployment capabilities given to engineers at OpenAI.
• At 55:44 - "One thing that I've learned... is how much of AI engineering is about building solutions to known limitations of the model. And then as the model gets better, you scrap that work and build new guardrails." - Janvi shares a key insight that much of her work involves creating engineering systems to work around the current imperfections of LLMs.

Takeaways

• Prioritize hands-on learning through hackathons and personal projects to stay current in the rapidly changing AI field, as traditional resources often lag behind.
• Use the "Product, Infrastructure, Model" framework to strategically navigate the AI job market, targeting companies that align with your specific career growth goals.
• Take initiative to build prototypes for user problems; a small hackathon project can evolve into a major product initiative and significantly accelerate your career.
• Master core software engineering skills like system design, debugging, and code comprehension, as these become even more critical when AI tools fail or complex systems break.
• When evaluating job offers in the volatile AI startup space, hold out for a role that is a definitive "heck yes" to ensure mutual commitment and alignment.
• Understand that a significant part of AI engineering involves building systems to compensate for the current limitations of AI models, and be prepared to discard and rebuild this work as models improve.