WebRTC: camera switch

Question

I want to be able to switch the camera in the middle of a call using WebRTC, without having to redo the call.

Suppose we have 2 mediaSources objects using the MediaStreamTrack.getSources method:

 { id: "id_source_1" | "id_source_2", facing: "user" | "environment", kind: "kind_1" | "kind_2", label: "label_1" | "label_2" } 

We start the call with "source_1" (referring to the "user"), and we want the user to be able to switch to "source_2" (before the "environment").

In my current code, when the user presses the "camera switch" button, the following is performed: ( callingSession - current WebRTC session)

 var mediaParams = { audio: true, video: { deviceId : source_2.id}, options: { muted: true, mirror: true }, elemId: 'localVideo' }; callingSession.getUserMedia(mediaParams, function (error, stream) { if (error) { console.error('error getting user media'); } else { var oldVideoTracks = callingSession.localStream.getVideoTracks(); var newVideoTracks = stream.getVideoTracks(); if (oldVideoTracks.length > 0 && newVideoTracks.length > 0) { callingSession.localStream.removeTrack(oldVideoTracks[0]); callingSession.localStream.addTrack(newVideoTracks[0]); } } }); 

As you can see, the mediaParams restrictions mediaParams now set to "source_2", we pass this mediaParams with the new restrictions to the getUserMedia method. Then we get the video tracks from both the old and the new stream.

The main problem in this code is that the old stream still exactly matches the new stream, even with the new restrictions passed to the getUserMedia method, so obviously the same video tracks, and of course, nothing happens, and the camera does not switch !!!

Am I doing something wrong in this code? Is there any way to switch the camera without re-arranging the call in WebRTC? what about the experimental applyConstraint() method I can't see it in chrome?

Thanks.

UPDATE My WebRTC application is an ionic application with crosswalk => chrome webview.

Answer 1

At the time of this writing, the WebRTC specification is very promising, but the implementation of this specification varies from browser to another. Currently, the Chrome implementation is still the same. However, thanks to arrow comments and to this SO answer, as well as a deeper understanding of SDP (Protocol session description) I can now switch the camera using Chrome.

At first, the restrictions on my getUserMedia method were incorrect, here's how I managed to pass the correct restrictions:

 var mediaParams = { // the other constraints video: {mandatory: {sourceId: source_2.id}} // ... }; 

After calling getUserMedia with the mediaParams argument mediaParams we need to remove the current stream from the peer, and then add a new one, for example:

 peerConnection.removeStream(peerConnection.getLocalStreams()[0]); peerConnection.addLocalStream(stream); 

These two lines of code will call onnegotiationneeded in the peerConnection object, which means that the peer must point to peer 2, it has changed the flow, so it needs a new description. This is why we need to create an offer, establish a new description and send this new description to the partner:

 peerConnection.createOffer() .then(function (offer) { peerConnection.setLocalDescription(offer); }) .then(function () { send(JSON.stringify({ "sdp": peerConnection.localDescription })); }); 

At the moment, it is up to you how you want to send SDP. (In my case, I had to send them using WebSockets.)

As soon as the other partner receives the new SDP, he must install it in his own peer-to-peer connection:

 var obj = JSON.parse(answer).sdp; peerConnection.setRemoteDescription(new RTCSessionDescription(obj)); 

I hope this one day helps someone.

Answer 2

The replaceTrack API was defined just for this.

In the end, chrome will support the RTCRtpSender.replaceTrack method ( http://w3c.imtqy.com/webrtc-pc/#rtcrtpsender-interface ), which can be used to replace the track without reconciling. You can track the development of this feature in Chrome here: https://www.chromestatus.com/feature/5347809238712320

It is already available to some extent in its own API. But it is currently being developed, so use at your own risk.