Bagisto, an open-source eCommerce platform powered by Laravel, is designed to be adaptable and robust, supporting businesses as they scale. As product catalogs grow into the millions, performance becomes critical.
This blog delves into advanced strategies for scaling Bagisto to efficiently manage 10 million products, focusing on key areas like database optimizations, caching solutions, and efficient queue management.
Understanding the Challenges
When scaling an eCommerce store to handle millions of products, the challenges evolve as the product catalog grows.
With around 0.2 million products, the system performs smoothly, handling database queries and searches efficiently without the need for advanced optimizations.
At 10 million products, the system experiences significant strain on the database and search, requiring advanced strategies like Elasticsearch, and optimized caching to maintain performance.
Key Challenges:
- Database Performance – Handling a massive volume of records efficiently without slowing down the system.
- Search Optimization – Ensuring that users can quickly find products, even with such a large catalog.
- Indexing & Caching – Reducing load times and improving response speed with proper indexing strategies and caching mechanisms.
Let’s explore how Bagisto, built on Laravel, handles these challenges and scales efficiently.
Database Performance in Bagisto
Bagisto uses MySQL/MariaDB by default, and to handle 10 million products efficiently, we need to optimize both the database and the way the platform interacts with it.
Optimizing Database Queries in Bagisto
Bagisto uses Eloquent ORM for database queries, which makes it easier to interact with the database but can sometimes lead to performance bottlenecks with large datasets. To mitigate this:
- Eager Loading: Avoid N+1 query problems by using eager loading for relationships. This ensures that related models are loaded in a single query, eliminating the need for multiple queries.
- Batch Updates: When updating large sets of records (e.g., updating product prices or inventory), use batch processing to handle them in chunks, reducing the load on the database.
Indexing Strategies in Bagisto for Scalable Database Performance
Bagisto structures indexed data for efficient search and retrieval, storing product information in Elasticsearch as documents within indices.
Each document in an index represents a product or product-related data (like product name, price, description, etc.).
This structured approach allows Elasticsearch to efficiently manage and retrieve product information.
When a user searches in Bagisto, Elasticsearch analyzes indexed data and retrieves relevant results. Its powerful algorithms ensure fast, accurate searches, enhancing the user experience.
To enhance Bagisto’s performance, the following indexers play a crucial role:
-
Price Indexing: Price indexing in Bagisto ensures accurate and up-to-date product prices across the storefront. It updates product prices in the database whenever changes occur, keeping storefront pricing consistent and reliable.
- Inventory Indexing: Inventory indexing in Bagisto keeps product stock levels accurate by updating quantities in real time. When products are bought, returned, or restocked, Bagisto automatically adjusts the inventory. This ensures the store always displays the correct stock and applies all pricing rules consistently.
- Flat Indexing: Flat indexing is a mechanism that optimizes product data retrieval. By processing products in batches, the system can handle large datasets efficiently without overloading the server. It’s particularly useful for managing large volumes of product attributes (e.g., SKU, price, weight) for various channels and locales.
- Catalog Rule Indexing: Catalog rule indexing updates product prices based on changes in catalog rules, such as expiring or modified offers. Bagisto schedules this indexing process daily at 00:01, ensuring that pricing remains accurate without requiring manual intervention.
-
ElasticSearch: Elasticsearch handles large data volumes and delivers fast, scalable search results. Bagisto uses Elasticsearch to index all products, enhancing search performance significantly.
Leveraging Elasticsearch for Scalable Search
One of the most critical components of scaling an eCommerce platform is ensuring that product search functionality remains fast and efficient, even with millions of products.
While MySQL/MariaDB can handle basic queries, they are not optimized for full-text search or complex filtering across large catalogs, which is where Elasticsearch excels. This is where Elasticsearch comes in.
1. Why Elasticsearch?
Elasticsearch is a distributed search engine optimized for speed and scalability, offering powerful full-text search capabilities. By integrating Elasticsearch with Bagisto, we can achieve the following benefits:
- Full-Text Search: Elasticsearch provides powerful full-text search capabilities, making it faster and more efficient than relational databases.
- Scalability: With its distributed architecture, Elasticsearch can scale horizontally by adding more nodes.
- Real-Time Indexing: When products are added, updated, or removed, Elasticsearch allows for real-time indexing. This ensures that product searches reflect the most up-to-date information, crucial for a dynamic e-commerce environment.
Integrating Elasticsearch with Bagisto enables search performance to scale seamlessly as product volume grows.
2. Integrating Elasticsearch with Bagisto
Integrating Elasticsearch with Bagisto is straightforward. First, configure your Elasticsearch instance and connect it through the admin panel. Bagisto will automatically index products, including attributes like name, description, price, and availability.
Here’s a simple configuration for integrating Elasticsearch with Bagisto:
To configure Elasticsearch, please refer to the Configuration Setup documentation.
3. Real-Time Indexing and Synchronization
Elasticsearch indexes products in real-time, ensuring that Bagisto instantly reflects additions or updates. This keeps product searches up-to-date, which is crucial in eCommerce environments where product information changes frequently.
- How Real-Time Indexing Works in Bagistos: When a product is updated in Bagisto—whether it’s a price change, stock update, or a new product—Elasticsearch quickly updates the search index.
This ensures customers always see the latest prices and availability in search results, improving their shopping experience. - Automatic Re-Indexing in Bagisto: Bagisto automatically updates Elasticsearch whenever products are added, updated, or deleted. This keeps search results accurate and in sync with the latest product data. Without this, users might see outdated prices or stock levels.
4. Final Performance Benchmark
Here’s a final benchmark of search performance and page load time after implementing Elasticsearch.
In the below image, you can see the page load time around 585.16 milliseconds with the 10 million products.
In the below image, you can see the Home page load time around 561 milliseconds with the 10 million products.
In the below image, you can see the category page load time around 516 milliseconds with the 10 million products.
Conclusion: Future Scalability Beyond 10 Million Products
Scaling Bagisto to handle 10 million products is a challenging but achievable task.
By optimizing the database with eager loading and batch updates, integrating Elasticsearch for fast, real-time search, and leveraging advanced caching techniques, you can ensure that your Bagisto-powered store remains performant, even as your product catalog grows.
Thank you for taking the time to read this guide. We hope it provided valuable insights for scaling your Bagisto store. If you have any questions or run into challenges, don’t hesitate to leave a comment we’re here to help!
If you’re looking for tailored solutions, consider hiring a Laravel developer to create custom features for your store. To discover more extensions and tools for Bagisto, explore the full range at bagisto.com.
