If you are intending to operate Jitsi Meet on a large scale to accommodate thousands of users, it is important to have the following:

01. Deploy multiple shards with HA proxy to ensure high availability and fault tolerance.

02. Implement load balancing for video bridges in each shard and utilize auto scaling to optimize costs based on demand.

03. Set up a pool of Jibris for recording or streaming purposes and utilize auto scaling to optimize costs as needed.

04. Establish a comprehensive monitoring system to track system load and conference metrics such as the number of conferences and users.

05. Implement a load testing mechanism to ensure that your infrastructure is ready for production, testing its performance under realistic conditions.

06. Configure Octo and turn servers to support multiple regions and, in specific cases, enable P2P conferences.

A Multi shard deployment with HA proxy

To efficiently handle a large user base in Jitsi Meet, it is recommended to distribute the load across multiple shards instead of relying on a single shard. Scaling the number of Jitsi Video Bridge instances within a single shard can lead to limitations due to potential bottlenecks in the XMPP server (typically Prosody) and Jicofo.

To implement this, deploy multiple shards of Jitsi Meet and utilize HAproxy with URL-based routing and stick tables. This configuration ensures that a specific conference room is consistently hosted on the same shard. For instance, if user A joins the conference room “myconference1” on shard1, all subsequent users joining “myconference1” will be directed to shard1 through the proxy.

By employing this setup, you can effectively distribute the load, optimize performance, and overcome potential limitations associated with scaling a single Jitsi Meet shard.

Load balanced Video Bridges in each shard with Auto Scaling

If you have numerous conferences running within a single shard, it is essential to distribute the load across multiple Jitsi Video Bridges for effective load balancing. Within a shard, Jicofo (Jitsi conference focus) takes charge of this load balancing once you configure multiple video bridges.

When hosting on a cloud platform like AWS, you can take advantage of auto scaling capabilities for the video bridges to handle dynamic loads in a cost-efficient manner. For example, during mid-day, there might be 500 concurrent users in the shard, whereas in the evening, there might only be 100 concurrent users. During nighttime, there may be no users at all. 

To optimize costs, the number of video bridges should automatically adjust to the optimal value based on the current load.

AWS offers tools like auto-scaling groups, SQS (Simple Queue Service), SNS (Simple Notification Service), and CloudWatch to facilitate this autoscaling process. Fine-tuning the autoscaling parameters for each specific use-case enables you to scale the video bridges up or down dynamically, aligning with the number of users and optimizing costs accordingly.

A collection of Jibri’s dedicated to recording or streaming purposes.

Jibri serves as the recording and streaming service for Jitsi, allowing one conference to be recorded per Jibri instance. To record multiple conferences simultaneously, an equivalent number of Jibri servers is required. However, maintaining a large pool of Jibris to accommodate the maximum expected number of simultaneous conferences can result in significant costs when hosted on AWS.

To address this challenge, we adopt a strategy that involves keeping a constant number of available Jibris (e.g., 3) at all times. When a user initiates a recording for a conference, one of the available Jibris is allocated for that specific conference, while another Jibri is automatically spun up to replenish the pool of free Jibris.

Once the recording is completed, the allocated Jibri can upload the recording to a storage service such as S3 before returning to the pool of free Jibris. If there are more than 3 available Jibris, the autoscaling mechanism can shut down one of the excess instances. The optimal number of constant Jibris is determined based on the specific use-case and requirements.

By implementing this approach, we strike a balance between efficient resource utilization and cost optimization, ensuring that the necessary recording capacity is available while avoiding unnecessary expenses associated with maintaining an excessive number of Jibri instances.

A load testing mechanism to evaluate the performance of your infrastructure.

Effective monitoring is essential for any system, including Jitsi. In addition to standard system metrics like CPU, memory, network, and IO, it is important to monitor specific aspects of Jitsi such as the number of users, conferences, video bridges, and Jibris. When utilizing AWS, CloudWatch can serve as the monitoring system of choice.

A mechanism for load testing to assess your infrastructure’s performance.

Before going live, it is crucial to conduct a load test to ensure that your infrastructure is correctly configured and capable of accommodating the desired number of simultaneous users. Jitsi Torture can be employed for this purpose, allowing you to simulate a high load on your infrastructure by generating fake conference users and flooding meetings. This load test helps evaluate the performance and scalability of your system prior to launching it into production.