Airbnbs JourneyFormer How Transformer Sequence Models Are Redefining Personalized Search

From Hand-Crafted Features to Learned Journeys

For years, Airbnb's search ranking relied on aggregated, hand-crafted statistics—like total past bookings or average listing price. While effective initially, this approach hit a scalability ceiling as feature counts grew into the hundreds, limiting the expressiveness of guest preference models. The core challenge? A guest's journey is a rich sequence of events: views, bookings, reviews, and cancellations spanning years. These events tell a story about what a guest truly values, but capturing that narrative with static features is like trying to understand a novel by only reading its table of contents.

This article explores how Airbnb's engineering team built JourneyFormer, a Transformer-based sequence model that encodes the full guest journey. This shift marks a significant evolution in recommender systems, moving from feature engineering to representation learning. For a broader look at how these platform changes impact developers, check out our coverage of the latest web platform updates.

The Architecture: Splitting Short-Term Intent from Long-Term Memory

A critical design decision was splitting the guest event sequence into two distinct parts to address computational and signal-sparsity challenges.

The Dual-Sequence Design

• Long-Term Sequence: Captures infrequent but highly informative events from the past seven years (bookings, reviews, cancellations). This sequence is capped at 80 events, providing depth and context.
• Short-Term Sequence: Captures listing views from the past 21 days, representing immediate browsing intent. This sequence is capped at 200 events, offering recency and immediate context.

Both sequences share a unified embedding table for high-cardinality IDs, like listing and host identifiers, allowing the model to learn rich, shared representations.

Efficiency: 4x Training Throughput

The team implemented three key strategies to make training on hundreds of millions of examples feasible:

1. Batching Searches: Instead of running the encoder for each search, the model runs once over the full event sequence. Due to the causal mask, intermediate embeddings are routed to corresponding searches, allowing multiple searches to share a single forward pass.
2. Length Bucketing: Sequences are grouped by length to minimize padding waste within batches.
3. Sparse Search Calculation: Padded searches are eliminated from the ranking model entirely.

# Simplified example of the batching strategy concept

def batch_search_encodings(event_sequence, search_timestamps):
    """
    Processes the full event sequence once and routes intermediate
    embeddings to corresponding searches.
    """
    # Placeholder for Transformer encoder forward pass
    all_embeddings = transformer_encoder(event_sequence)

    search_embeddings = {}
    for timestamp in search_timestamps:
        # The embedding at timestamp-1 captures all events up to that point
        search_embeddings[timestamp] = all_embeddings[timestamp - 1]

    return search_embeddings

Serving: Decoupling Inference for Low Latency

To keep query-time latency low, the system decouples inference into two stages:

• Daily Batch Job: The sequence encoder processes event history daily and stores the resulting embeddings.
• Real-time Retrieval: At search time, the ranking model retrieves the stored embedding and combines it with the live search query to score candidate listings.

Results, Limitations, and the Road Ahead

The results are impressive. A/B tests showed statistically significant gains: +0.55% uncanceled bookers and +0.90% views when combining both sequences. The final setwise ranker contributed an additional +0.28% uncanceled bookings. The complete system achieved a +3.78% offline NDCG improvement—a substantial gain in a system refined over a decade.

Limitations and Considerations

While powerful, this approach has limitations:

• Computational Cost: Training a Transformer on this scale requires significant infrastructure investment.
• Cold Start Problem: The model relies on historical events, making it less effective for new guests with sparse histories.
• Data Recency: The daily batch inference means the model may not reflect changes in guest intent within a single day.

Next Steps for Learning

To dive deeper into this domain, explore:

1. Setwise Ranking: Learn how it differs from pointwise ranking by considering a set of candidates together.
2. Sequence Modeling: Study the Transformer architecture and its applications beyond NLP.
3. Embedding Learning: Understand how unified embedding tables handle high-cardinality features.

Conclusion: A New Era of Personalized Search

JourneyFormer represents a fundamental shift in how Airbnb understands its guests. By learning directly from the full arc of guest behavior, the model captures nuanced preferences that were previously inaccessible. The success across both search and promotional emails validates the power and generality of this approach. This is a clear signal that the future of recommender systems lies not in more features, but in better representations of user journeys. For more on this topic, see our guide on styling search highlight pseudo-elements for front-end insights.

Source: Airbnb Engineering Blog