Enhancing Conversational AI for E-Commerce with Bottom-Up Synthesis

Enhancing Conversational AI for E-Commerce with Bottom-Up Synthesis

Kun Qian¹, Maximillian Chen¹, Siyan Li¹, Arpit Sharma², Zhou Yu¹³

1: Columbia University 2: Walmart 3: Arklex.ai

Introduction

Building high-quality conversational AI systems requires vast amounts of domain-specific training data. However, collecting real-world task-oriented dialogues can be challenging due to data scarcity and privacy concerns. Traditional methods rely on a top-down approach, where large language models (LLMs) generate entire conversations from a broad prompt. While this method ensures fluency, it lacks control over finer details, leading to inaccuracies or hallucinations.

To address these challenges, researchers have introduced Bottom-Up Conversation Synthesis (BUSY)—a novel framework that constructs realistic task-oriented dialogues by first generating question-answer (QA) pairs and then assembling them into a coherent conversation. This bottom-up methodology offers improved control, factual accuracy, and privacy compliance, making it particularly valuable for knowledge-grounded applications like e-commerce chatbots.

In this article, we explore how the BUSY framework is revolutionizing synthetic dialogue generation, with a focus on its application to the Walmart Shopping Companion dataset.

Limitations of Traditional Top-Down Dialogue Generation

The conventional approach to generating synthetic conversations follows a top-down methodology:

1. A high-level prompt is provided to an LLM.
2. The model generates a multi-turn dialogue in a single pass.
3. The resulting conversation is used for training chatbots.

While this method is effective in producing fluent interactions, it has notable limitations:

• Lack of Granularity: A single broad prompt cannot enforce fine-grained control over each turn in the conversation.
• Hallucinations: The model may generate incorrect or irrelevant information, especially when external knowledge is required.
• Privacy Concerns: When dealing with proprietary databases, non-local models require access to confidential data, raising security issues.

To overcome these shortcomings, the BUSY framework takes a bottom-up approach, ensuring higher-quality dialogues with greater control.

Bottom-Up Conversation Synthesis (BUSY) Explained

BUSY follows a structured, stepwise process to generate task-oriented and knowledge-grounded conversations. Instead of synthesizing an entire dialogue in one go, it first generates question-answer pairs and then constructs full conversations by linking them logically.

Step 1: Question Generation

BUSY begins by generating realistic customer questions in three phases:

1. Extracting Attributes: The model analyzes real user questions and product databases to identify key attributes (e.g., product specifications, features, pricing).
2. Iterative Refinement: LLMs generate questions based on extracted attributes, and prompts are refined over multiple iterations to improve realism.
3. Validation: The generated questions are compared against human-written queries to ensure alignment.

💡 Example: Instead of prompting an LLM to generate an entire conversation about a vacuum cleaner, BUSY first generates focused questions like:

• “What is the battery life of this vacuum?”
• “Is this model suitable for cleaning pet hair?”

Step 2: Answer Generation

Once the questions are created, answers are generated using a structured database rather than relying on free-text generation. This ensures factual correctness and prevents hallucinations.

• Answers are strictly retrieved from a verified product knowledge base.
• If an answer is unavailable, the model responds with a predefined template (e.g., “I’m sorry, this information is not provided. Do you want to contact the provider?”).
• All answers are generated locally to protect data privacy.

💡 Example:

Q: Does this vacuum come with a HEPA filter?

A: Yes, this model includes a HEPA-certified filter for allergen control.

Step 3: Connecting QA Pairs into Conversations

With high-quality QA pairs in place, the final step is to stitch them together into full-fledged dialogues:

• A virtual assistant persona is added to maintain natural flow.
• Opening, closing, and transition turns are inserted to create a seamless conversation.
• Dialogues include realistic user behaviors, such as frustration when information is missing.

💡 Example Dialogue Snippet:

Evaluating BUSY: How Does It Perform?

To validate BUSY, the researchers generated 6,000 dialogues for various e-commerce categories (e.g., vacuums, diapers, clothing). Both human evaluators and LLMs were used to assess dialogue quality.

Key Findings

• Higher Truthfulness: Since answers are database-grounded, hallucinations are significantly reduced.
• Improved User Experience: Conversations are more structured and informative compared to traditional top-down approaches.
• Better Control & Adaptability: Iterative prompt refinement allows fine-tuning of generated data, leading to more natural and domain-specific interactions.

Comparison with Top-Down Generation:

Takeaway: BUSY offers a more realistic, controllable, and privacy-friendly approach to dialogue generation.

Real-World Applications of BUSY

BUSY is particularly useful for knowledge-grounded applications, including:

• E-Commerce Assistants 🛒 (e.g., Walmart Shopping Companion)
• Customer Support Chatbots 💬 (handling FAQs with factual responses)
• Healthcare AI Assistants 🏥 (providing evidence-based medical guidance)
• Technical Support Bots 🔧 (offering precise troubleshooting steps)

By ensuring fact-based responses, BUSY enhances the reliability and trustworthiness of AI-powered assistants.

Conclusion & Future Directions

The BUSY framework represents a major leap forward in synthetic dialogue generation. By shifting from top-down to bottom-up synthesis, it delivers higher accuracy, better control, and improved privacy.

What’s next?

• Expanding BUSY beyond e-commerce to healthcare, finance, and education.
• Enhancing context awareness to handle long-form multi-turn conversations.
• Integrating reinforcement learning to fine-tune dialogue coherence.

As AI-powered assistants continue to evolve, bottom-up synthesis could become the new standard for generating factually grounded and user-friendly conversations.

Try it out and give our open-source project a star:

https://github.com/arklexai/Agent-First-Organization