How Perplexity Builds Its Product

In yesterday’s article about the creator economy, I mentioned Lenny, the author of Lenny’s Newsletter, who said that all the advertising slots for his podcast this year have already been sold out, which shows how popular his podcast is.

Today, I share his recent article “How Perplexity Builds Product,” in which Perplexity’s co-founder and head of product, Johnny Ho, shares some experiences they had while building the Perplexity product. The process is quite similar to how Notion built Notion AI, as discussed in “How Notion Built Notion AI.”

Perplexity is arguably the hottest AI startup right now, with a team of fewer than 50 people and a valuation of $1 billion. Just two weeks ago, it was reported that they are undergoing a new round of financing of at least $250 million, with a valuation expected to be between $2.5 billion and $3 billion. The recently launched Enterprise Pro version may be creating a new market.

There are a few points in this article that left a deep impression on me, for example:

1. There are very few product managers, so they tend to hire self-driven ICs and deliberately avoid hiring those who are best at guiding others’ work;

2. The most difficult and important part of a product manager’s job is having taste in application scenarios; technical project managers or engineers with product taste will become the most valuable people in the company;

3. Small team operations, with a typical Perplexity team consisting of 2-3 people, the recently launched AI podcast was built and operated by just one person;

4. Organizing like slime molds, optimizing by parallelizing as many projects as possible to minimize coordination costs;

5. In terms of roadmap construction, high-level goals and directions are top-down, but many new ideas are bottom-up;

6. Many of Perplexity’s features and products are built in hackathons lasting a week (or less);

With the help of AI capabilities, I did a simple translation of the article to share:

1. How to Use AI Tools to Build Perplexity?

To be honest, at first, we didn’t know how to do various things, including product management, project management, finance, human resources, etc. We gained access to GPT-3 early on, and when we were exploring how to establish the company, we would start everything by asking the AI, “What is X?” “How should we do X correctly?”

For example, we would ask questions like, “How to launch a product? What steps should be involved in the launch?” This way, you would get a rough step-by-step process, which is good enough for a startup. Obviously, the first attempt is usually not right, but humans are the same, right? So we could only iterate from there naturally.

Trying to solve problems entirely on our own would take days, but with AI and some prompts, we could start working in five minutes. We are still doing this. For example, this week, I asked Perplexity: “How do I write an email inviting someone to use Perplexity Pro?”

We even sometimes try to use it to build our products, but we found that AI tools are not good enough for programming. It can help us write scripts, but if you want sustainable code to build a platform, it doesn’t really work. Even today, with technological advancements and the latest models, it still only writes templates. You can’t really use it to design a new, long-lasting abstraction.

2. How Many PMs Do You Have?

In our team of 50 people, there are only 2 full-time product managers (as shown in the image below). Most projects involve only one or two people, and the most challenging projects have at most 3-4 people. For example, our AI podcast was produced from start to finish by one person, who is a brand designer, but he also does audio engineering and conducts various research to figure out how to create the most interactive and engaging podcast. I believe the project manager was never involved in this process.

Product management is only utilized when a very difficult decision needs to be made and involves multiple directions or more projects. The most difficult and important part of a product manager’s job is having taste in application scenarios.

For AI, there may be too many potential application scenarios to develop. Therefore, product managers must make qualitative decisions based on data, user research, etc. For example, a significant issue faced by AI is how to prioritize between productivity-focused application scenarios and more engaging chatbot-type application scenarios. We decided early on to focus on the former but are still having ongoing discussions.

We plan to hire one or two more project managers next year, but the hiring threshold will remain high.

3. What Do You Value Most When Hiring (Maybe Others Don’t)?

Given our pace of work, what we value most is flexibility and proactivity. For us, the most important thing is to be able to constructively work in a resource-limited environment (which may require wearing multiple hats).

When you look at a PM’s resume, many of them prioritize helping others and seeking consensus. I think, with the emergence of AI, this has become less important.

Therefore, you don’t necessarily need to have skills in processing processes or leading teams as before. We are looking for strong individual contributors who have a significant quantitative impact on users, rather than influence within the company. If I see terms like “Agile Expert” or “Scrum Master” on a resume, it might not be a good fit.

Moreover, AI allows PMs to do more IC work, especially in data analysis and customer insights. Of course, you still need some foundational knowledge (basic math, statistics, and programming), but it has never been easier to be a truly “technical” project manager.

We still choose people who fit the culture and are easy to collaborate with, but we rarely look for those who guide others’ work, as this is not so necessary for AI. As the scale increases, this may change, but at the current scale, the number of products that need to be developed far exceeds the number of people who can be投入其中.

In the future, I expect management layers across the industry to decrease. If I had to guess, over time, technical project managers or engineers with product taste will become the most valuable people in the company.

4. Do You Build Teams Around Products, User Types, User Journeys, Outcomes, or Something In Between? Has This Changed Over the Years?

My goal is to build teams around the concept of minimizing “coordination friction,” as described by Alex Komoroske in his PPT about viewing organizations as slime molds. The general idea is that coordination costs (caused by uncertainty and disagreement) increase with scale, and adding more managers does not improve the situation.

People’s motivations can misalign, and people tend to lie to their managers, who then lie to their managers. If you want to communicate with someone in another part of the organization, you have to go up two levels and down two levels, asking everyone along the way.

Instead, what you want to do is keep overall goals aligned and conduct parallel projects directed towards that goal by sharing reusable guidelines and processes. Especially with the development of AI, using AI to perform “rubber duck debugging” on ideas, rather than relying on perfect consistency and consensus, can minimize coordination costs.

We also updated a list of “who is who” in internal documentation, so if you feel the need to contact anyone, just reach out directly. This requires a high level of trust.

But more importantly, with AI, you don’t have to interact with people as often. Before asking others, you can take a minute to ask AI to reduce coordination costs and provide everyone with a reasonable starting point to work on their own.

5. How Long Is Your Detailed Planning, and How Has It Evolved Over the Years?

Perplexity has been established for less than two years, and the situation in the AI field changes too rapidly to make commitments two years out. We create quarterly plans, and within the quarter, we strive to maintain stability in the product roadmap. The roadmap contains several well-known large projects and some smaller tasks, which we adjust based on changing priorities.

Flexibility is crucial, as developments in AI often have unforeseen impacts. For example, rapid advancements in open-source models and context lengths have affected products, roadmaps, and overall business. Recently, Meta released Llama 3, and Mistral released 8x22B; we are looking for creative ways to incorporate these models into our products.

Projects in the product roadmap also need to be flexible, as new product development runs parallel to technical/model development roadmaps. Engineers switch between maintaining existing products and developing new products based on weekly conditions. When we encounter limitations in existing systems and accumulate technical debt, the technical roadmap tends to grow quickly, but we prioritize addressing technical debt that can lead to product improvements.

However, within a given week, plans are quite stable. Every week, we hold a kickoff meeting where everyone sets high-level expectations for their week. We have a culture of setting 75% weekly goals: everyone identifies their top priorities for the week and strives to achieve 75% of them by the weekend. Just listing a few points ensures that priorities are clear within the week.

Taking some time at the beginning of each week to reflect on meta-tasks can bring clarity and prevent overreacting or busy decision-making. Over time, our ability to estimate scale and prioritize based on return on investment has also improved.

6. Do You Have Any Form of OKRs?

We strive to be as rigorous and data-driven as possible in our quarterly planning. All goals are measurable, whether through quantifiable thresholds or boolean values indicating “Is X completed?” Our goals are very ambitious, and usually by the end of the quarter, we only complete 70% in one direction or another.

The remaining 30% helps identify gaps in prioritization and personnel allocation. For example, when infrastructure goals are not met, the issue of insufficient investment in hiring infrastructure engineers quickly becomes apparent.

7. How Do Your Product/Design Review Meetings Work?

Once central goals and high-level designs are determined, projects are driven by a single DRI, executing steps as parallel as possible.

The first step of any project is to break it down into parallel tasks as much as possible to reduce coordination issues. We do this in linear projects, and I lead this work with the project manager in the team (or the person responsible for project management duties). We strive for each task to be independent, and you should be able to execute tasks without hindrance. You will likely need to make some controversial decisions, but you can resolve those controversies later.

At the start of each project, there is a quick coordination kickoff, followed by asynchronous iterations without constraints or review processes. When individuals feel they can provide feedback on design, implementation, or the final product, they share it on Slack, and other team members provide honest and constructive feedback. Iterations happen naturally as needed, and products are only launched after they have garnered enough attention through internal use.

I encourage everyone to work in parallel as much as possible. They should not wait for everyone to unlock. Ideally, design, frontend, and backend work simultaneously on the same project. Now that we have a business team, all four can work in parallel, whereas traditionally, you might have to wait for design or models to come first.

8. How Do Reporting Relationships Work?

Currently, the team is built functionally (product, R&D, design, business, etc.), with different teams considering different aspects of the company and stack, but all efforts focus on improving the core product. The goals we design can be transformed into universal top-level metrics to comprehensively improve the user experience. For example, all teams share common top-level metrics when conducting A/B testing within their stack layers. Given the rapid changes in products, we want to avoid political issues where anyone’s identity is tied to any given component of the product.

At our current scale, our design is flat, and the reporting structure does not determine priorities as much as commitment to top-level goals. Our two full-time product managers (one for the web and one for mobile) report to me as product leads. We find that when teams do not have a product manager, team members take on product manager responsibilities, such as adjusting scope, making user-facing decisions, and trusting their own taste.

9. Having Built One of the Most Popular and Successful Products, What Unique Aspects or Core Factors Do You Think Led to Its Success?

The core of our approach is listening to feedback from users and internally and distilling it into intuitive products that apply to many customers. We also try to distill feedback in a way that motivates and informs our team, setting a broad vision while allowing individuals to control their own decisions about what best serves the initial goals.

Our decentralized decision-making approach passes the torch of responsibility, allowing for rapid iteration without approval processes. Individuals make urgent, locally optimal decisions. Any misalignment is quickly resolved.

10. What Are the Main Tools You Use for Task Management and Bug Tracking?

We use Linear. For AI products, the boundaries between tasks, bugs, and projects become blurred, but we find that many concepts in Linear (such as leads, classifications, resizing, etc.) are very important. One of my favorite features is automatic archiving; if a task hasn’t been mentioned for a while, it may not actually be important.

The main tool we use to store facts like roadmaps and milestone plans is Notion. We use Notion for design documents and RFCs during the development process, and then for documentation, post-mortems, and historical records. Writing ideas down (recording the chain of thought) can make decision-making clearer and make it easier to adjust asynchronously and avoid meetings.

Unwrap.ai is a tool we recently introduced to integrate, record, and quantify qualitative feedback. Due to the nature of AI, many questions do not always have enough certainty to be classified as bugs. Grouping individual feedback into more specific themes and areas for improvement.

11. Do You Think Roadmap Ideas Are Primarily Top-Down (Teams Are Told What to Build) or Bottom-Up (Teams Usually Propose Ideas)?

High-level goals and directions are top-down, but many new ideas are bottom-up. We firmly believe that engineering and design should own ideas and details, especially for AI products, where constraints are unknown before turning ideas into code and models. A lot of brainstorming is ongoing. We have a dedicated brainstorming channel in Slack, and subsequent ideas are collected in Linear, usually refined directly without anyone asking.

The best case for bottom-up ideas can be seen in Perplexity’s Discovery, Collection, and sharing experiences. For example, as I shared above, our brand designer Phi built the Discover Daily podcast and made decisions around scripting, ElevenLabs integration, branding, and audio engineering simultaneously. With AI, the application scenarios could not be predicted before product iteration releases. A year ago, we never anticipated that the experience of the “Discovery” module would ultimately be embedded into the podcast.

12. When People See Companies Like Yours From the Outside, Everything Seems Perfect, Like You’ve Figured Everything Out. What Are Some Things That Don’t Work Well or Face Huge Challenges?

Today, the biggest challenge is scaling from our current size to a new level, whether in hiring or execution and planning. We don’t want to lose our core identity of working in a very flat and collaborative environment. Even small decisions, like how to organize Slack and Linear, become difficult to scale. We are currently trying to figure out how to maintain transparency while expanding the number of channels and projects without leading to notification explosions.

13. Do You Have Any Interesting/Unique Rituals or Traditions in Your Product Team or Company?

Many features and products of Perplexity are built in hackathons lasting a week (or less) (which is similar to how Notion initially built its AI products). It turns out that focused sprints to build new features are the most exciting and memorable moments. Our first interactive search prototype, Pro Search (formerly known as Copilot), was built in just a few days, but it was improved through multiple iterations and refinements.