Skip to main content

Behind Billions of Calls: Enterprise-Level Stability Assurance of Bella OpenAPI

Introduction: The Gap Between Experiment and Production​

In today's rapidly developing AI technology landscape, many enterprises face a common dilemma when building AI applications: open-source projects often perform well in experimental environments but struggle to meet the stringent stability and performance requirements of enterprise-level applications in real production environments. Particularly when daily call volumes reach millions, tens of millions, or even hundreds of millions, system stability issues tend to emerge all at once.

The Bella OpenAPI project stands out in this regard—it is not a proof of concept born in a laboratory, but an enterprise-grade AI capability gateway battle-tested across all Beike's business lines, handling up to 150 million API calls daily. In this article, we will unveil how Bella OpenAPI has built a stability assurance system capable of supporting such scale.

Data Speaks: The Real Test of 150 Million Daily Calls​

The Value of Scale Validation​

Unlike most similar open-source projects, every component and design decision in Bella OpenAPI has been rigorously tested in Beike's production environment:

  • Average daily calls: 150 million API requests
  • Business coverage: All Beike business lines, including property search, customer service, content creation, and many other scenarios
  • Service reliability: 99.99% service availability
  • Concurrent processing capability: Supporting tens of thousands of QPS during peak periods

Behind these numbers are the results of countless system optimizations and architectural adjustments, which serve as the most powerful proof of Bella OpenAPI's distinction from other open-source AI gateways.

Architecture Decoded: The Technical Foundation Supporting Billions of Calls​

Multi-level Architectural Design​

Bella OpenAPI adopts a battle-tested multi-layer architecture:

  • Access layer: High-performance gateway based on Nginx, implementing preliminary request distribution and security filtering
  • Service layer: Microservice architecture with independently deployed capability modules for fault isolation
  • Storage layer: Combination of MySQL and Redis, balancing data persistence and access speed
  • Monitoring layer: Comprehensive monitoring and alerting system to promptly detect and resolve potential issues

This layered design ensures high availability and scalability of the system, allowing the overall service to remain operational even when individual components fail.

Key Technologies for High Concurrency Processing​

The following key technologies played crucial roles in handling billions of calls:

1. Async Processing Framework Based on Disruptor​

Bella OpenAPI employs the high-performance Disruptor ring buffer to implement asynchronous processing of log events:

  • Lock-free design: Reduces thread contention and improves throughput
  • Multi-processor parallelism: Simultaneously processes tasks such as billing, metrics collection, and rate limiting
  • Elegant back-pressure handling: Automatically adjusts processing speed when system load is high

Tests show that this design improves the system's log processing capability by nearly 10 times under high load, while significantly reducing the risk of main business thread blocking.

2. Multi-level Cache Architecture​

A multi-level cache design combining Redisson, Caffeine, and JetCache:

  • Local cache: Reduces network overhead and accelerates reading of frequently accessed data
  • Distributed cache: Ensures data consistency in cluster environments
  • Hierarchical invalidation strategy: Designs different caching strategies based on data characteristics

This cache architecture allows over 90% of metadata access to be retrieved directly from cache in high-concurrency scenarios, significantly reducing database pressure.

3. Efficient Distributed Rate Limiting Mechanism​

Distributed rate limiting implementation based on Redis+Lua:

  • Atomic operations: Uses Lua scripts to ensure atomicity of rate limit judgment and counting
  • Sliding window algorithm: More precise traffic control than fixed windows
  • Multi-level rate limiting strategy: Multiple safeguards at API, user, and service levels

This rate limiting mechanism can respond to rate limit judgments for a large number of concurrent requests at the millisecond level, successfully defending against system risks brought by multiple traffic surges.

Stability Assurance: Not Just Technology, But a System​

Comprehensive Monitoring and Early Warning​

No matter how excellent the system design, long-term stable operation cannot be guaranteed without comprehensive monitoring. Bella OpenAPI has built a multi-dimensional monitoring system:

  • Business metrics monitoring: API call volume, success rate, latency distribution, etc.
  • System resource monitoring: CPU, memory, network, disk, and other basic resource usage
  • Dependency service monitoring: Health status of databases, caches, and third-party AI services
  • Exception event monitoring: Error logs, exception stacks, slow queries, and other anomalies

These monitoring data are not only used for real-time alerts but also provide valuable data support for system performance optimization.

Intelligent Degradation and Circuit Breaking Strategies​

Facing inevitable service fluctuations, Bella OpenAPI has implemented fine-grained degradation and circuit breaking strategies:

  • Service-level circuit breaking: Automatically switches to backup channels when an AI service provider experiences issues
  • Function-level degradation: Prioritizes the availability of core functions under extreme loads
  • Resource isolation: Prevents single service failures from affecting the global system through resource pool isolation

During a critical third-party model service interruption event, these mechanisms caused Bella OpenAPI's service availability to drop by only 0.2%, while applications calling directly were completely interrupted.

Canary Releases and End-to-End Testing​

Every system update goes through a strict release process:

  • Multi-environment validation: Layer-by-layer verification from development environment to testing environment to pre-release environment
  • Canary releases: New versions are first validated in a small scope before being fully released
  • End-to-end pressure testing: Pressure tests simulating real traffic to verify the performance of various system components
  • Rollback plans: Detailed rollback solutions are prepared for each release to ensure quick recovery if problems arise

This set of processes ensures that Bella OpenAPI maintains a high level of stability even during frequent iterations.

Practical Insights: Lessons Learned from Bella OpenAPI​

1. Stability is Built in Layers​

System stability is not a single-point breakthrough but a multi-level, comprehensive system construction:

  • Architecture layer: Distributed, microservices, fault isolation
  • Technology layer: High-performance components, asynchronous processing, multi-level caching
  • Operations layer: Monitoring alerts, canary releases, emergency plans
  • Process layer: End-to-end testing, performance evaluation, continuous optimization

2. Balance Between Performance and Stability​

Pursuing extreme performance often sacrifices stability, while being overly conservative affects user experience. Bella OpenAPI's experience is:

  • Performance targets should have reasonable expectations with sufficient resource margin
  • Stability measures themselves need to be high-performance and should not become new bottlenecks
  • Find the optimal balance between performance and stability through fine-grained monitoring

3. Start Small, Expand Gradually​

Even a system like Bella OpenAPI that supports billions of calls started small and expanded and optimized gradually:

  • First validate in non-core businesses, then expand to core scenarios after accumulating experience
  • Continuously monitor system bottlenecks and optimize architecture based on real data
  • Maintain architectural elasticity to adapt to growing business requirements

Conclusion: Stability is the Foundation of AI Applications​

While pursuing AI technological innovation, we should not forget that stable and reliable infrastructure is the prerequisite for all innovative applications.

Bella OpenAPI has proven its value in the field of enterprise-level AI infrastructure through the practical test of 150 million daily calls. Whether building an internal AI capability platform or providing AI services externally, Bella OpenAPI offers a battle-tested reliable starting point. This is also the original intention of open-sourcing this project—to share battle-tested enterprise-level solutions and promote technological progress throughout the industry.

For teams eager to quickly build enterprise-level AI applications, Bella OpenAPI provides a validated shortcut: instead of building infrastructure from scratch, stand on the shoulders of giants and focus on creating business value.

If you are interested in Bella OpenAPI's stability assurance mechanisms, we welcome you to visit our GitHub repository or directly experience our deployed online version.