With Kopano Meet. we decided to bring WebRTC calls and conferences to mobile devices. This means that in addition to all the desktop platforms, we needed to support Apple and Android devices in a self-hosted environment. The following article is an overview of the basic problems and challenges we encountered when we implemented Kopano meet.
Meet supports three different WebRTC implementations – Chromium from Google, Firefox from Mozilla and Safari from Apple.
Meet provides full interoperability with a single progressive web app implementation on the desktop (Windows, MacOS and Linux) and mobile (Android and iOS). This means that the differences between the environments which can run Meet have to be handled gracefully.
Camera and microphone access
To ensure minimal bandwidth usage and also to allow mobile devices to enter power saving modes, accessing camera and microphone hardware must only be done on demand.
To further reduce bandwidth usage when the camera and/or microphone is disabled, we wanted to release the hardware access for devices not in use. Doing so required to add/remove tracks from the WebRTC connection as the user interacts with the application.
In previous implementations, we just muted and unmuted tracks. Meet removes tracks (when a certain media gets muted) and adds back new tracks (as required) when unmuted. This also allows seamless change of video resolution and even switching cameras (for example front and back camera).
Mute and unmute of the corresponding tracks received from a device might automatically release access to the underlying hardware in some platforms, but will still send empty stream which is undesirable for bandwidth and CPU usage. Thus Kopano Meet truly disables the camera/microphone when muted to achieve the best privacy control possible.
WebRTC 1.0 implementation woes with streams/tracks
WebRTC implementations still have not fully implemented WebRTC 1.0 as defined in its standard. Especially Chromium which was the first to provide reasonably usable WebRTC support is at the time of writing is switching over its SDP format from Chromium-specific “Plan B” to the standards conformant unified format in January 2019. Since Meet can use multiple streams of the same type, Meet keeps using the “Plan B” implementation for a while longer until support for older Chromiums (which only support “Plan B” SDP) can be removed.
Once all supported WebRTC implementations use the unified plan, the track add/remove behavior can be considered to be enabled for all vendors where it is currently only enabled for Chromium.
Network connectivity is not guaranteed
It is certainly not news that network connectivity can be flaky – this is even more so in a mobile environment. Devices switch between mobile and Wi-Fi, go through tunnels with no reception or loose cell tower connectivity while on the move.
To handle this, solid detection and recovery of connection errors is required. Meet use a “no matter what approach” – if an unrecoverable connection error is encountered, the old and potentially broken connection is replaced by a new one.
Many connections – full mesh
Establishing a connection between two peers is easy enough. Since Kopano Meet also supports conferences or multi-user groups, Meet uses a full mesh topology for small groups, where every peer establishes a fully end-to-end encrypted connection to every other peer.
This means a client has to handle n – 1 number of connections, all with their individual latency and potentially individual connection errors between peers.
To detect issues with peer connectivity and to be able to throw away broken connections on both ends quickly, all the signaling runs through a Kopano Web Meetings server which adds additional WebRTC connection identity on the fly, enabling the client to make educated decisions if a connection should be killed or if it still might recover.
The Apple case
Kopano Meet also supports Apple’s WebRTC implementation since the iOS platform does not allow any other WebRTC engine to exist in web browsers. Because Kopano is 100 percent self-hosted, the browser is the environment where self-hosted apps are running. Apple is late with supporting WebRTC at all and the state of their implementation is not very great.
Playing multiple videos on Apple – huh?
It turned out that Safari simply cannot play more than on video element which also has an unmuted audio stream – and that is by design.
This is a problem for any conference/group call. I think this is nuts – especially for streams coming in from a peer connection. Thus workaround time.
Kopano Meet solves this problem by playing all remote video streams without sound while adding the soundtrack to an additional invisible audio element. Did I mention nuts?
All things considered, I guess one can be grateful that WebRTC made it into Apple devices at all, but for the time being Apple users might encounter lip-sync issues because of this.
The Microsoft case
So Microsoft has this browser called Edge (see market share at statcounter. Now that Microsoft is switching their browser Engine to Chromium and Edge as of now has numerous issues with WebRTC 1.0 interoperability, we decided from the start that Edge was not a platform which we needed to support.
On Windows, every user has the choice to use Firefox or one of the numerous Chromium-based browsers like Iridium and over time Edge might be automatically supported via its Chromium-based engine.
Implementing Meet as a progressive web app with WebRTC support on essentially all relevant browser platforms turned out to work well. WebRTC still evolves as the browser vendors catch up with the specification but we are able to handle this gracefully via adapters and the occasional small workaround.
To install your own self-hosted Kopano Meet follow the installation instructions in the Kopano Meet manual.
Now that we have an end-to-end encrypted call and conference application which can be self-hosted by our customers without having to roll an app to any of the mobile vendor’s stores the next steps are to allow guest access, screen sharing and call notifications. This will utilize additional promising new web standards like Screen Share and Web Push and.
Stay tuned to the Kopano blog for news and updates on Kopano Meet.