/* jshint -W117 */ var TraceablePeerConnection = require("./TraceablePeerConnection"); var SDPDiffer = require("./SDPDiffer"); var SDPUtil = require("./SDPUtil"); var SDP = require("./SDP"); var async = require("async"); var transform = require("sdp-transform"); var XMPPEvents = require("../../service/xmpp/XMPPEvents"); var RTCBrowserType = require("../RTC/RTCBrowserType"); // Jingle stuff function JingleSession(me, sid, connection, service, eventEmitter) { this.me = me; this.sid = sid; this.connection = connection; this.initiator = null; this.responder = null; this.isInitiator = null; this.peerjid = null; this.state = null; this.localSDP = null; this.remoteSDP = null; this.relayedStreams = []; this.startTime = null; this.stopTime = null; this.media_constraints = null; this.pc_constraints = null; this.ice_config = {}; this.drip_container = []; this.service = service; this.eventEmitter = eventEmitter; this.usetrickle = true; this.usepranswer = false; // early transport warmup -- mind you, this might fail. depends on webrtc issue 1718 this.usedrip = false; // dripping is sending trickle candidates not one-by-one this.hadstuncandidate = false; this.hadturncandidate = false; this.lasticecandidate = false; this.statsinterval = null; this.reason = null; this.addssrc = []; this.removessrc = []; this.pendingop = null; this.switchstreams = false; this.wait = true; this.localStreamsSSRC = null; this.ssrcOwners = {}; this.ssrcVideoTypes = {}; this.eventEmitter = eventEmitter; /** * The indicator which determines whether the (local) video has been muted * in response to a user command in contrast to an automatic decision made * by the application logic. */ this.videoMuteByUser = false; this.modifySourcesQueue = async.queue(this._modifySources.bind(this), 1); // We start with the queue paused. We resume it when the signaling state is // stable and the ice connection state is connected. this.modifySourcesQueue.pause(); } JingleSession.prototype.updateModifySourcesQueue = function() { var signalingState = this.peerconnection.signalingState; var iceConnectionState = this.peerconnection.iceConnectionState; if (signalingState === 'stable' && iceConnectionState === 'connected') { this.modifySourcesQueue.resume(); } else { this.modifySourcesQueue.pause(); } }; JingleSession.prototype.initiate = function (peerjid, isInitiator) { var self = this; if (this.state !== null) { console.error('attempt to initiate on session ' + this.sid + 'in state ' + this.state); return; } this.isInitiator = isInitiator; this.state = 'pending'; this.initiator = isInitiator ? this.me : peerjid; this.responder = !isInitiator ? this.me : peerjid; this.peerjid = peerjid; this.hadstuncandidate = false; this.hadturncandidate = false; this.lasticecandidate = false; this.peerconnection = new TraceablePeerConnection( this.connection.jingle.ice_config, this.connection.jingle.pc_constraints, this); this.peerconnection.onicecandidate = function (event) { self.sendIceCandidate(event.candidate); }; this.peerconnection.onaddstream = function (event) { if (event.stream.id !== 'default') { console.log("REMOTE STREAM ADDED: ", event.stream , event.stream.id); self.remoteStreamAdded(event); } else { // This is a recvonly stream. Clients that implement Unified Plan, // such as Firefox use recvonly "streams/channels/tracks" for // receiving remote stream/tracks, as opposed to Plan B where there // are only 3 channels: audio, video and data. console.log("RECVONLY REMOTE STREAM IGNORED: " + event.stream + " - " + event.stream.id); } }; this.peerconnection.onremovestream = function (event) { // Remove the stream from remoteStreams // FIXME: remotestreamremoved.jingle not defined anywhere(unused) $(document).trigger('remotestreamremoved.jingle', [event, self.sid]); }; this.peerconnection.onsignalingstatechange = function (event) { if (!(self && self.peerconnection)) return; self.updateModifySourcesQueue(); }; this.peerconnection.oniceconnectionstatechange = function (event) { if (!(self && self.peerconnection)) return; self.updateModifySourcesQueue(); switch (self.peerconnection.iceConnectionState) { case 'connected': this.startTime = new Date(); break; case 'disconnected': this.stopTime = new Date(); break; case 'failed': self.eventEmitter.emit(XMPPEvents.CONFERENCE_SETUP_FAILED); break; } onIceConnectionStateChange(self.sid, self); }; this.peerconnection.onnegotiationneeded = function (event) { self.eventEmitter.emit(XMPPEvents.PEERCONNECTION_READY, self); }; // add any local and relayed stream APP.RTC.localStreams.forEach(function(stream) { self.peerconnection.addStream(stream.getOriginalStream()); }); this.relayedStreams.forEach(function(stream) { self.peerconnection.addStream(stream); }); }; function onIceConnectionStateChange(sid, session) { switch (session.peerconnection.iceConnectionState) { case 'checking': session.timeChecking = (new Date()).getTime(); session.firstconnect = true; break; case 'completed': // on caller side case 'connected': if (session.firstconnect) { session.firstconnect = false; var metadata = {}; metadata.setupTime = (new Date()).getTime() - session.timeChecking; session.peerconnection.getStats(function (res) { if(res && res.result) { res.result().forEach(function (report) { if (report.type == 'googCandidatePair' && report.stat('googActiveConnection') == 'true') { metadata.localCandidateType = report.stat('googLocalCandidateType'); metadata.remoteCandidateType = report.stat('googRemoteCandidateType'); // log pair as well so we can get nice pie // charts metadata.candidatePair = report.stat('googLocalCandidateType') + ';' + report.stat('googRemoteCandidateType'); if (report.stat('googRemoteAddress').indexOf('[') === 0) { metadata.ipv6 = true; } } }); } }); } break; } } JingleSession.prototype.getVideoType = function () { return APP.desktopsharing.isUsingScreenStream() ? 'screen' : 'camera'; }; JingleSession.prototype.accept = function () { this.state = 'active'; var pranswer = this.peerconnection.localDescription; if (!pranswer || pranswer.type != 'pranswer') { return; } console.log('going from pranswer to answer'); if (this.usetrickle) { // remove candidates already sent from session-accept var lines = SDPUtil.find_lines(pranswer.sdp, 'a=candidate:'); for (var i = 0; i < lines.length; i++) { pranswer.sdp = pranswer.sdp.replace(lines[i] + '\r\n', ''); } } while (SDPUtil.find_line(pranswer.sdp, 'a=inactive')) { // FIXME: change any inactive to sendrecv or whatever they were originally pranswer.sdp = pranswer.sdp.replace('a=inactive', 'a=sendrecv'); } var prsdp = new SDP(pranswer.sdp); var accept = $iq({to: this.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: 'session-accept', initiator: this.initiator, responder: this.responder, sid: this.sid }); // FIXME why do we generate session-accept in 3 different places ? prsdp.toJingle( accept, this.initiator == this.me ? 'initiator' : 'responder', this.localStreamsSSRC, self.getVideoType()); var sdp = this.peerconnection.localDescription.sdp; while (SDPUtil.find_line(sdp, 'a=inactive')) { // FIXME: change any inactive to sendrecv or whatever they were originally sdp = sdp.replace('a=inactive', 'a=sendrecv'); } var self = this; this.peerconnection.setLocalDescription(new RTCSessionDescription({type: 'answer', sdp: sdp}), function () { //console.log('setLocalDescription success'); self.setLocalDescription(); self.connection.sendIQ(accept, function () { var ack = {}; ack.source = 'answer'; $(document).trigger('ack.jingle', [self.sid, ack]); }, function (stanza) { var error = ($(stanza).find('error').length) ? { code: $(stanza).find('error').attr('code'), reason: $(stanza).find('error :first')[0].tagName }:{}; error.source = 'answer'; JingleSession.onJingleError(self.sid, error); }, 10000); }, function (e) { console.error('setLocalDescription failed', e); self.eventEmitter.emit(XMPPEvents.CONFERENCE_SETUP_FAILED); } ); }; JingleSession.prototype.terminate = function (reason) { this.state = 'ended'; this.reason = reason; this.peerconnection.close(); if (this.statsinterval !== null) { window.clearInterval(this.statsinterval); this.statsinterval = null; } }; JingleSession.prototype.active = function () { return this.state == 'active'; }; JingleSession.prototype.sendIceCandidate = function (candidate) { var self = this; if (candidate && !this.lasticecandidate) { var ice = SDPUtil.iceparams(this.localSDP.media[candidate.sdpMLineIndex], this.localSDP.session); var jcand = SDPUtil.candidateToJingle(candidate.candidate); if (!(ice && jcand)) { console.error('failed to get ice && jcand'); return; } ice.xmlns = 'urn:xmpp:jingle:transports:ice-udp:1'; if (jcand.type === 'srflx') { this.hadstuncandidate = true; } else if (jcand.type === 'relay') { this.hadturncandidate = true; } if (this.usetrickle) { if (this.usedrip) { if (this.drip_container.length === 0) { // start 20ms callout window.setTimeout(function () { if (self.drip_container.length === 0) return; self.sendIceCandidates(self.drip_container); self.drip_container = []; }, 20); } this.drip_container.push(candidate); return; } else { self.sendIceCandidate([candidate]); } } } else { //console.log('sendIceCandidate: last candidate.'); if (!this.usetrickle) { //console.log('should send full offer now...'); //FIXME why do we generate session-accept in 3 different places ? var init = $iq({to: this.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: this.peerconnection.localDescription.type == 'offer' ? 'session-initiate' : 'session-accept', initiator: this.initiator, sid: this.sid}); this.localSDP = new SDP(this.peerconnection.localDescription.sdp); var self = this; var sendJingle = function (ssrc) { if(!ssrc) ssrc = {}; self.localSDP.toJingle( init, self.initiator == self.me ? 'initiator' : 'responder', ssrc, self.getVideoType()); self.connection.sendIQ(init, function () { //console.log('session initiate ack'); var ack = {}; ack.source = 'offer'; $(document).trigger('ack.jingle', [self.sid, ack]); }, function (stanza) { self.state = 'error'; self.peerconnection.close(); var error = ($(stanza).find('error').length) ? { code: $(stanza).find('error').attr('code'), reason: $(stanza).find('error :first')[0].tagName, }:{}; error.source = 'offer'; JingleSession.onJingleError(self.sid, error); }, 10000); } sendJingle(); } this.lasticecandidate = true; console.log('Have we encountered any srflx candidates? ' + this.hadstuncandidate); console.log('Have we encountered any relay candidates? ' + this.hadturncandidate); if (!(this.hadstuncandidate || this.hadturncandidate) && this.peerconnection.signalingState != 'closed') { $(document).trigger('nostuncandidates.jingle', [this.sid]); } } }; JingleSession.prototype.sendIceCandidates = function (candidates) { console.log('sendIceCandidates', candidates); var cand = $iq({to: this.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: 'transport-info', initiator: this.initiator, sid: this.sid}); for (var mid = 0; mid < this.localSDP.media.length; mid++) { var cands = candidates.filter(function (el) { return el.sdpMLineIndex == mid; }); var mline = SDPUtil.parse_mline(this.localSDP.media[mid].split('\r\n')[0]); if (cands.length > 0) { var ice = SDPUtil.iceparams(this.localSDP.media[mid], this.localSDP.session); ice.xmlns = 'urn:xmpp:jingle:transports:ice-udp:1'; cand.c('content', {creator: this.initiator == this.me ? 'initiator' : 'responder', name: (cands[0].sdpMid? cands[0].sdpMid : mline.media) }).c('transport', ice); for (var i = 0; i < cands.length; i++) { cand.c('candidate', SDPUtil.candidateToJingle(cands[i].candidate)).up(); } // add fingerprint if (SDPUtil.find_line(this.localSDP.media[mid], 'a=fingerprint:', this.localSDP.session)) { var tmp = SDPUtil.parse_fingerprint(SDPUtil.find_line(this.localSDP.media[mid], 'a=fingerprint:', this.localSDP.session)); tmp.required = true; cand.c( 'fingerprint', {xmlns: 'urn:xmpp:jingle:apps:dtls:0'}) .t(tmp.fingerprint); delete tmp.fingerprint; cand.attrs(tmp); cand.up(); } cand.up(); // transport cand.up(); // content } } // might merge last-candidate notification into this, but it is called alot later. See webrtc issue #2340 //console.log('was this the last candidate', this.lasticecandidate); this.connection.sendIQ(cand, function () { var ack = {}; ack.source = 'transportinfo'; $(document).trigger('ack.jingle', [this.sid, ack]); }, function (stanza) { var error = ($(stanza).find('error').length) ? { code: $(stanza).find('error').attr('code'), reason: $(stanza).find('error :first')[0].tagName, }:{}; error.source = 'transportinfo'; JingleSession.onJingleError(this.sid, error); }, 10000); }; JingleSession.prototype.sendOffer = function () { //console.log('sendOffer...'); var self = this; this.peerconnection.createOffer(function (sdp) { self.createdOffer(sdp); }, function (e) { console.error('createOffer failed', e); }, this.media_constraints ); }; // FIXME createdOffer is never used in jitsi-meet JingleSession.prototype.createdOffer = function (sdp) { //console.log('createdOffer', sdp); var self = this; this.localSDP = new SDP(sdp.sdp); //this.localSDP.mangle(); var sendJingle = function () { var init = $iq({to: this.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: 'session-initiate', initiator: this.initiator, sid: this.sid}); self.localSDP.toJingle( init, this.initiator == this.me ? 'initiator' : 'responder', this.localStreamsSSRC, self.getVideoType()); self.connection.sendIQ(init, function () { var ack = {}; ack.source = 'offer'; $(document).trigger('ack.jingle', [self.sid, ack]); }, function (stanza) { self.state = 'error'; self.peerconnection.close(); var error = ($(stanza).find('error').length) ? { code: $(stanza).find('error').attr('code'), reason: $(stanza).find('error :first')[0].tagName, }:{}; error.source = 'offer'; JingleSession.onJingleError(self.sid, error); }, 10000); } sdp.sdp = this.localSDP.raw; this.peerconnection.setLocalDescription(sdp, function () { if(self.usetrickle) { sendJingle(); } self.setLocalDescription(); //console.log('setLocalDescription success'); }, function (e) { console.error('setLocalDescription failed', e); self.eventEmitter.emit(XMPPEvents.CONFERENCE_SETUP_FAILED); } ); var cands = SDPUtil.find_lines(this.localSDP.raw, 'a=candidate:'); for (var i = 0; i < cands.length; i++) { var cand = SDPUtil.parse_icecandidate(cands[i]); if (cand.type == 'srflx') { this.hadstuncandidate = true; } else if (cand.type == 'relay') { this.hadturncandidate = true; } } }; JingleSession.prototype.readSsrcInfo = function (contents) { var self = this; $(contents).each(function (idx, content) { var name = $(content).attr('name'); var mediaType = this.getAttribute('name'); var ssrcs = $(content).find('description>source[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"]'); ssrcs.each(function () { var ssrc = this.getAttribute('ssrc'); $(this).find('>ssrc-info[xmlns="http://jitsi.org/jitmeet"]').each( function () { var owner = this.getAttribute('owner'); var videoType = this.getAttribute('video-type'); self.ssrcOwners[ssrc] = owner; self.ssrcVideoTypes[ssrc] = videoType; } ); }); }); }; JingleSession.prototype.getSsrcOwner = function (ssrc) { return this.ssrcOwners[ssrc]; }; JingleSession.prototype.setRemoteDescription = function (elem, desctype) { //console.log('setting remote description... ', desctype); this.remoteSDP = new SDP(''); this.remoteSDP.fromJingle(elem); this.readSsrcInfo($(elem).find(">content")); if (this.peerconnection.remoteDescription !== null) { console.log('setRemoteDescription when remote description is not null, should be pranswer', this.peerconnection.remoteDescription); if (this.peerconnection.remoteDescription.type == 'pranswer') { var pranswer = new SDP(this.peerconnection.remoteDescription.sdp); for (var i = 0; i < pranswer.media.length; i++) { // make sure we have ice ufrag and pwd if (!SDPUtil.find_line(this.remoteSDP.media[i], 'a=ice-ufrag:', this.remoteSDP.session)) { if (SDPUtil.find_line(pranswer.media[i], 'a=ice-ufrag:', pranswer.session)) { this.remoteSDP.media[i] += SDPUtil.find_line(pranswer.media[i], 'a=ice-ufrag:', pranswer.session) + '\r\n'; } else { console.warn('no ice ufrag?'); } if (SDPUtil.find_line(pranswer.media[i], 'a=ice-pwd:', pranswer.session)) { this.remoteSDP.media[i] += SDPUtil.find_line(pranswer.media[i], 'a=ice-pwd:', pranswer.session) + '\r\n'; } else { console.warn('no ice pwd?'); } } // copy over candidates var lines = SDPUtil.find_lines(pranswer.media[i], 'a=candidate:'); for (var j = 0; j < lines.length; j++) { this.remoteSDP.media[i] += lines[j] + '\r\n'; } } this.remoteSDP.raw = this.remoteSDP.session + this.remoteSDP.media.join(''); } } var remotedesc = new RTCSessionDescription({type: desctype, sdp: this.remoteSDP.raw}); this.peerconnection.setRemoteDescription(remotedesc, function () { //console.log('setRemoteDescription success'); }, function (e) { console.error('setRemoteDescription error', e); JingleSession.onJingleFatalError(self, e); } ); }; JingleSession.prototype.addIceCandidate = function (elem) { var self = this; if (this.peerconnection.signalingState == 'closed') { return; } if (!this.peerconnection.remoteDescription && this.peerconnection.signalingState == 'have-local-offer') { console.log('trickle ice candidate arriving before session accept...'); // create a PRANSWER for setRemoteDescription if (!this.remoteSDP) { var cobbled = 'v=0\r\n' + 'o=- ' + '1923518516' + ' 2 IN IP4 0.0.0.0\r\n' +// FIXME 's=-\r\n' + 't=0 0\r\n'; // first, take some things from the local description for (var i = 0; i < this.localSDP.media.length; i++) { cobbled += SDPUtil.find_line(this.localSDP.media[i], 'm=') + '\r\n'; cobbled += SDPUtil.find_lines(this.localSDP.media[i], 'a=rtpmap:').join('\r\n') + '\r\n'; if (SDPUtil.find_line(this.localSDP.media[i], 'a=mid:')) { cobbled += SDPUtil.find_line(this.localSDP.media[i], 'a=mid:') + '\r\n'; } cobbled += 'a=inactive\r\n'; } this.remoteSDP = new SDP(cobbled); } // then add things like ice and dtls from remote candidate elem.each(function () { for (var i = 0; i < self.remoteSDP.media.length; i++) { if (SDPUtil.find_line(self.remoteSDP.media[i], 'a=mid:' + $(this).attr('name')) || self.remoteSDP.media[i].indexOf('m=' + $(this).attr('name')) === 0) { if (!SDPUtil.find_line(self.remoteSDP.media[i], 'a=ice-ufrag:')) { var tmp = $(this).find('transport'); self.remoteSDP.media[i] += 'a=ice-ufrag:' + tmp.attr('ufrag') + '\r\n'; self.remoteSDP.media[i] += 'a=ice-pwd:' + tmp.attr('pwd') + '\r\n'; tmp = $(this).find('transport>fingerprint'); if (tmp.length) { self.remoteSDP.media[i] += 'a=fingerprint:' + tmp.attr('hash') + ' ' + tmp.text() + '\r\n'; } else { console.log('no dtls fingerprint (webrtc issue #1718?)'); self.remoteSDP.media[i] += 'a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:BAADBAADBAADBAADBAADBAADBAADBAADBAADBAAD\r\n'; } break; } } } }); this.remoteSDP.raw = this.remoteSDP.session + this.remoteSDP.media.join(''); // we need a complete SDP with ice-ufrag/ice-pwd in all parts // this makes the assumption that the PRANSWER is constructed such that the ice-ufrag is in all mediaparts // but it could be in the session part as well. since the code above constructs this sdp this can't happen however var iscomplete = this.remoteSDP.media.filter(function (mediapart) { return SDPUtil.find_line(mediapart, 'a=ice-ufrag:'); }).length == this.remoteSDP.media.length; if (iscomplete) { console.log('setting pranswer'); try { this.peerconnection.setRemoteDescription(new RTCSessionDescription({type: 'pranswer', sdp: this.remoteSDP.raw }), function() { }, function(e) { console.log('setRemoteDescription pranswer failed', e.toString()); }); } catch (e) { console.error('setting pranswer failed', e); } } else { //console.log('not yet setting pranswer'); } } // operate on each content element elem.each(function () { // would love to deactivate this, but firefox still requires it var idx = -1; var i; for (i = 0; i < self.remoteSDP.media.length; i++) { if (SDPUtil.find_line(self.remoteSDP.media[i], 'a=mid:' + $(this).attr('name')) || self.remoteSDP.media[i].indexOf('m=' + $(this).attr('name')) === 0) { idx = i; break; } } if (idx == -1) { // fall back to localdescription for (i = 0; i < self.localSDP.media.length; i++) { if (SDPUtil.find_line(self.localSDP.media[i], 'a=mid:' + $(this).attr('name')) || self.localSDP.media[i].indexOf('m=' + $(this).attr('name')) === 0) { idx = i; break; } } } var name = $(this).attr('name'); // TODO: check ice-pwd and ice-ufrag? $(this).find('transport>candidate').each(function () { var line, candidate; line = SDPUtil.candidateFromJingle(this); candidate = new RTCIceCandidate({sdpMLineIndex: idx, sdpMid: name, candidate: line}); try { self.peerconnection.addIceCandidate(candidate); } catch (e) { console.error('addIceCandidate failed', e.toString(), line); } }); }); }; JingleSession.prototype.sendAnswer = function (provisional) { //console.log('createAnswer', provisional); var self = this; this.peerconnection.createAnswer( function (sdp) { self.createdAnswer(sdp, provisional); }, function (e) { console.error('createAnswer failed', e); self.eventEmitter.emit(XMPPEvents.CONFERENCE_SETUP_FAILED); }, this.media_constraints ); }; JingleSession.prototype.createdAnswer = function (sdp, provisional) { //console.log('createAnswer callback'); var self = this; this.localSDP = new SDP(sdp.sdp); //this.localSDP.mangle(); this.usepranswer = provisional === true; if (this.usetrickle) { if (this.usepranswer) { sdp.type = 'pranswer'; for (var i = 0; i < this.localSDP.media.length; i++) { this.localSDP.media[i] = this.localSDP.media[i].replace('a=sendrecv\r\n', 'a=inactive\r\n'); } this.localSDP.raw = this.localSDP.session + '\r\n' + this.localSDP.media.join(''); } } var self = this; var sendJingle = function (ssrcs) { // FIXME why do we generate session-accept in 3 different places ? var accept = $iq({to: self.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: 'session-accept', initiator: self.initiator, responder: self.responder, sid: self.sid }); self.localSDP.toJingle( accept, self.initiator == self.me ? 'initiator' : 'responder', ssrcs, self.getVideoType()); self.connection.sendIQ(accept, function () { var ack = {}; ack.source = 'answer'; $(document).trigger('ack.jingle', [self.sid, ack]); }, function (stanza) { var error = ($(stanza).find('error').length) ? { code: $(stanza).find('error').attr('code'), reason: $(stanza).find('error :first')[0].tagName, }:{}; error.source = 'answer'; JingleSession.onJingleError(self.sid, error); }, 10000); } sdp.sdp = this.localSDP.raw; this.peerconnection.setLocalDescription(sdp, function () { //console.log('setLocalDescription success'); if (self.usetrickle && !self.usepranswer) { sendJingle(); } self.setLocalDescription(); }, function (e) { console.error('setLocalDescription failed', e); self.eventEmitter.emit(XMPPEvents.CONFERENCE_SETUP_FAILED); } ); var cands = SDPUtil.find_lines(this.localSDP.raw, 'a=candidate:'); for (var j = 0; j < cands.length; j++) { var cand = SDPUtil.parse_icecandidate(cands[j]); if (cand.type == 'srflx') { this.hadstuncandidate = true; } else if (cand.type == 'relay') { this.hadturncandidate = true; } } }; JingleSession.prototype.sendTerminate = function (reason, text) { var self = this, term = $iq({to: this.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: 'session-terminate', initiator: this.initiator, sid: this.sid}) .c('reason') .c(reason || 'success'); if (text) { term.up().c('text').t(text); } this.connection.sendIQ(term, function () { self.peerconnection.close(); self.peerconnection = null; self.terminate(); var ack = {}; ack.source = 'terminate'; $(document).trigger('ack.jingle', [self.sid, ack]); }, function (stanza) { var error = ($(stanza).find('error').length) ? { code: $(stanza).find('error').attr('code'), reason: $(stanza).find('error :first')[0].tagName, }:{}; $(document).trigger('ack.jingle', [self.sid, error]); }, 10000); if (this.statsinterval !== null) { window.clearInterval(this.statsinterval); this.statsinterval = null; } }; JingleSession.prototype.addSource = function (elem, fromJid) { var self = this; // FIXME: dirty waiting if (!this.peerconnection.localDescription) { console.warn("addSource - localDescription not ready yet") setTimeout(function() { self.addSource(elem, fromJid); }, 200 ); return; } console.log('addssrc', new Date().getTime()); console.log('ice', this.peerconnection.iceConnectionState); this.readSsrcInfo(elem); var sdp = new SDP(this.peerconnection.remoteDescription.sdp); var mySdp = new SDP(this.peerconnection.localDescription.sdp); $(elem).each(function (idx, content) { var name = $(content).attr('name'); var lines = ''; $(content).find('ssrc-group[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"]').each(function() { var semantics = this.getAttribute('semantics'); var ssrcs = $(this).find('>source').map(function () { return this.getAttribute('ssrc'); }).get(); if (ssrcs.length != 0) { lines += 'a=ssrc-group:' + semantics + ' ' + ssrcs.join(' ') + '\r\n'; } }); var tmp = $(content).find('source[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"]'); // can handle both >source and >description>source tmp.each(function () { var ssrc = $(this).attr('ssrc'); if(mySdp.containsSSRC(ssrc)){ /** * This happens when multiple participants change their streams at the same time and * ColibriFocus.modifySources have to wait for stable state. In the meantime multiple * addssrc are scheduled for update IQ. See */ console.warn("Got add stream request for my own ssrc: "+ssrc); return; } if (sdp.containsSSRC(ssrc)) { console.warn("Source-add request for existing SSRC: " + ssrc); return; } $(this).find('>parameter').each(function () { lines += 'a=ssrc:' + ssrc + ' ' + $(this).attr('name'); if ($(this).attr('value') && $(this).attr('value').length) lines += ':' + $(this).attr('value'); lines += '\r\n'; }); }); sdp.media.forEach(function(media, idx) { if (!SDPUtil.find_line(media, 'a=mid:' + name)) return; sdp.media[idx] += lines; if (!self.addssrc[idx]) self.addssrc[idx] = ''; self.addssrc[idx] += lines; }); sdp.raw = sdp.session + sdp.media.join(''); }); this.modifySourcesQueue.push(function() { // When a source is added and if this is FF, a new channel is allocated // for receiving the added source. We need to diffuse the SSRC of this // new recvonly channel to the rest of the peers. console.log('modify sources done'); var newSdp = new SDP(self.peerconnection.localDescription.sdp); console.log("SDPs", mySdp, newSdp); self.notifyMySSRCUpdate(mySdp, newSdp); }); }; JingleSession.prototype.removeSource = function (elem, fromJid) { var self = this; // FIXME: dirty waiting if (!this.peerconnection.localDescription) { console.warn("removeSource - localDescription not ready yet") setTimeout(function() { self.removeSource(elem, fromJid); }, 200 ); return; } console.log('removessrc', new Date().getTime()); console.log('ice', this.peerconnection.iceConnectionState); var sdp = new SDP(this.peerconnection.remoteDescription.sdp); var mySdp = new SDP(this.peerconnection.localDescription.sdp); $(elem).each(function (idx, content) { var name = $(content).attr('name'); var lines = ''; $(content).find('ssrc-group[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"]').each(function() { var semantics = this.getAttribute('semantics'); var ssrcs = $(this).find('>source').map(function () { return this.getAttribute('ssrc'); }).get(); if (ssrcs.length != 0) { lines += 'a=ssrc-group:' + semantics + ' ' + ssrcs.join(' ') + '\r\n'; } }); var tmp = $(content).find('source[xmlns="urn:xmpp:jingle:apps:rtp:ssma:0"]'); // can handle both >source and >description>source tmp.each(function () { var ssrc = $(this).attr('ssrc'); // This should never happen, but can be useful for bug detection if(mySdp.containsSSRC(ssrc)){ console.error("Got remove stream request for my own ssrc: "+ssrc); return; } $(this).find('>parameter').each(function () { lines += 'a=ssrc:' + ssrc + ' ' + $(this).attr('name'); if ($(this).attr('value') && $(this).attr('value').length) lines += ':' + $(this).attr('value'); lines += '\r\n'; }); }); sdp.media.forEach(function(media, idx) { if (!SDPUtil.find_line(media, 'a=mid:' + name)) return; sdp.media[idx] += lines; if (!self.removessrc[idx]) self.removessrc[idx] = ''; self.removessrc[idx] += lines; }); sdp.raw = sdp.session + sdp.media.join(''); }); this.modifySourcesQueue.push(function() { // When a source is removed and if this is FF, the recvonly channel that // receives the remote stream is deactivated . We need to diffuse the // recvonly SSRC removal to the rest of the peers. console.log('modify sources done'); var newSdp = new SDP(self.peerconnection.localDescription.sdp); console.log("SDPs", mySdp, newSdp); self.notifyMySSRCUpdate(mySdp, newSdp); }); }; JingleSession.prototype._modifySources = function (successCallback, queueCallback) { var self = this; if (this.peerconnection.signalingState == 'closed') return; if (!(this.addssrc.length || this.removessrc.length || this.pendingop !== null || this.switchstreams)){ // There is nothing to do since scheduled job might have been executed by another succeeding call this.setLocalDescription(); if(successCallback){ successCallback(); } queueCallback(); return; } // Reset switch streams flag this.switchstreams = false; var sdp = new SDP(this.peerconnection.remoteDescription.sdp); // add sources this.addssrc.forEach(function(lines, idx) { sdp.media[idx] += lines; }); this.addssrc = []; // remove sources this.removessrc.forEach(function(lines, idx) { lines = lines.split('\r\n'); lines.pop(); // remove empty last element; lines.forEach(function(line) { sdp.media[idx] = sdp.media[idx].replace(line + '\r\n', ''); }); }); this.removessrc = []; sdp.raw = sdp.session + sdp.media.join(''); this.peerconnection.setRemoteDescription(new RTCSessionDescription({type: 'offer', sdp: sdp.raw}), function() { if(self.signalingState == 'closed') { console.error("createAnswer attempt on closed state"); queueCallback("createAnswer attempt on closed state"); return; } self.peerconnection.createAnswer( function(modifiedAnswer) { // change video direction, see https://github.com/jitsi/jitmeet/issues/41 if (self.pendingop !== null) { var sdp = new SDP(modifiedAnswer.sdp); if (sdp.media.length > 1) { switch(self.pendingop) { case 'mute': sdp.media[1] = sdp.media[1].replace('a=sendrecv', 'a=recvonly'); break; case 'unmute': sdp.media[1] = sdp.media[1].replace('a=recvonly', 'a=sendrecv'); break; } sdp.raw = sdp.session + sdp.media.join(''); modifiedAnswer.sdp = sdp.raw; } self.pendingop = null; } // FIXME: pushing down an answer while ice connection state // is still checking is bad... //console.log(self.peerconnection.iceConnectionState); // trying to work around another chrome bug //modifiedAnswer.sdp = modifiedAnswer.sdp.replace(/a=setup:active/g, 'a=setup:actpass'); self.peerconnection.setLocalDescription(modifiedAnswer, function() { //console.log('modified setLocalDescription ok'); self.setLocalDescription(); if(successCallback){ successCallback(); } queueCallback(); }, function(error) { console.error('modified setLocalDescription failed', error); queueCallback(error); } ); }, function(error) { console.error('modified answer failed', error); queueCallback(error); } ); }, function(error) { console.error('modify failed', error); queueCallback(error); } ); }; /** * Switches video streams. * @param new_stream new stream that will be used as video of this session. * @param oldStream old video stream of this session. * @param success_callback callback executed after successful stream switch. */ JingleSession.prototype.switchStreams = function (new_stream, oldStream, success_callback, isAudio) { var self = this; // Remember SDP to figure out added/removed SSRCs var oldSdp = null; if(self.peerconnection) { if(self.peerconnection.localDescription) { oldSdp = new SDP(self.peerconnection.localDescription.sdp); } self.peerconnection.removeStream(oldStream, true); if(new_stream) self.peerconnection.addStream(new_stream); } // Conference is not active if(!oldSdp || !self.peerconnection) { success_callback(); return; } self.switchstreams = true; self.modifySourcesQueue.push(function() { console.log('modify sources done'); success_callback(); var newSdp = new SDP(self.peerconnection.localDescription.sdp); console.log("SDPs", oldSdp, newSdp); self.notifyMySSRCUpdate(oldSdp, newSdp); }); }; /** * Figures out added/removed ssrcs and send update IQs. * @param old_sdp SDP object for old description. * @param new_sdp SDP object for new description. */ JingleSession.prototype.notifyMySSRCUpdate = function (old_sdp, new_sdp) { if (!(this.peerconnection.signalingState == 'stable' && this.peerconnection.iceConnectionState == 'connected')){ console.log("Too early to send updates"); return; } // send source-remove IQ. sdpDiffer = new SDPDiffer(new_sdp, old_sdp); var remove = $iq({to: this.peerjid, type: 'set'}) .c('jingle', { xmlns: 'urn:xmpp:jingle:1', action: 'source-remove', initiator: this.initiator, sid: this.sid } ); var removed = sdpDiffer.toJingle(remove); if (removed) { this.connection.sendIQ(remove, function (res) { console.info('got remove result', res); }, function (err) { console.error('got remove error', err); } ); } else { console.log('removal not necessary'); } // send source-add IQ. var sdpDiffer = new SDPDiffer(old_sdp, new_sdp); var add = $iq({to: this.peerjid, type: 'set'}) .c('jingle', { xmlns: 'urn:xmpp:jingle:1', action: 'source-add', initiator: this.initiator, sid: this.sid } ); var added = sdpDiffer.toJingle(add, this.getVideoType()); if (added) { this.connection.sendIQ(add, function (res) { console.info('got add result', res); }, function (err) { console.error('got add error', err); } ); } else { console.log('addition not necessary'); } }; /** * Mutes/unmutes the (local) video i.e. enables/disables all video tracks. * * @param mute true to mute the (local) video i.e. to disable all video * tracks; otherwise, false * @param callback a function to be invoked with mute after all video * tracks have been enabled/disabled. The function may, optionally, return * another function which is to be invoked after the whole mute/unmute operation * has completed successfully. * @param options an object which specifies optional arguments such as the * boolean key byUser with default value true which * specifies whether the method was initiated in response to a user command (in * contrast to an automatic decision made by the application logic) */ JingleSession.prototype.setVideoMute = function (mute, callback, options) { var byUser; if (options) { byUser = options.byUser; if (typeof byUser === 'undefined') { byUser = true; } } else { byUser = true; } // The user's command to mute the (local) video takes precedence over any // automatic decision made by the application logic. if (byUser) { this.videoMuteByUser = mute; } else if (this.videoMuteByUser) { return; } this.hardMuteVideo(mute); var self = this; var oldSdp = null; if(self.peerconnection) { if(self.peerconnection.localDescription) { oldSdp = new SDP(self.peerconnection.localDescription.sdp); } } this.modifySourcesQueue.push(function() { console.log('modify sources done'); callback(mute); var newSdp = new SDP(self.peerconnection.localDescription.sdp); console.log("SDPs", oldSdp, newSdp); self.notifyMySSRCUpdate(oldSdp, newSdp); }); }; JingleSession.prototype.hardMuteVideo = function (muted) { this.pendingop = muted ? 'mute' : 'unmute'; }; JingleSession.prototype.sendMute = function (muted, content) { var info = $iq({to: this.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: 'session-info', initiator: this.initiator, sid: this.sid }); info.c(muted ? 'mute' : 'unmute', {xmlns: 'urn:xmpp:jingle:apps:rtp:info:1'}); info.attrs({'creator': this.me == this.initiator ? 'creator' : 'responder'}); if (content) { info.attrs({'name': content}); } this.connection.send(info); }; JingleSession.prototype.sendRinging = function () { var info = $iq({to: this.peerjid, type: 'set'}) .c('jingle', {xmlns: 'urn:xmpp:jingle:1', action: 'session-info', initiator: this.initiator, sid: this.sid }); info.c('ringing', {xmlns: 'urn:xmpp:jingle:apps:rtp:info:1'}); this.connection.send(info); }; JingleSession.prototype.getStats = function (interval) { var self = this; var recv = {audio: 0, video: 0}; var lost = {audio: 0, video: 0}; var lastrecv = {audio: 0, video: 0}; var lastlost = {audio: 0, video: 0}; var loss = {audio: 0, video: 0}; var delta = {audio: 0, video: 0}; this.statsinterval = window.setInterval(function () { if (self && self.peerconnection && self.peerconnection.getStats) { self.peerconnection.getStats(function (stats) { var results = stats.result(); // TODO: there are so much statistics you can get from this.. for (var i = 0; i < results.length; ++i) { if (results[i].type == 'ssrc') { var packetsrecv = results[i].stat('packetsReceived'); var packetslost = results[i].stat('packetsLost'); if (packetsrecv && packetslost) { packetsrecv = parseInt(packetsrecv, 10); packetslost = parseInt(packetslost, 10); if (results[i].stat('googFrameRateReceived')) { lastlost.video = lost.video; lastrecv.video = recv.video; recv.video = packetsrecv; lost.video = packetslost; } else { lastlost.audio = lost.audio; lastrecv.audio = recv.audio; recv.audio = packetsrecv; lost.audio = packetslost; } } } } delta.audio = recv.audio - lastrecv.audio; delta.video = recv.video - lastrecv.video; loss.audio = (delta.audio > 0) ? Math.ceil(100 * (lost.audio - lastlost.audio) / delta.audio) : 0; loss.video = (delta.video > 0) ? Math.ceil(100 * (lost.video - lastlost.video) / delta.video) : 0; $(document).trigger('packetloss.jingle', [self.sid, loss]); }); } }, interval || 3000); return this.statsinterval; }; JingleSession.onJingleError = function (session, error) { console.error("Jingle error", error); } JingleSession.onJingleFatalError = function (session, error) { this.service.sessionTerminated = true; this.connection.emuc.doLeave(); this.eventEmitter.emit(XMPPEvents.CONFERENCE_SETUP_FAILED); this.eventEmitter.emit(XMPPEvents.JINGLE_FATAL_ERROR, session, error); } JingleSession.prototype.setLocalDescription = function () { var self = this; var newssrcs = []; var session = transform.parse(this.peerconnection.localDescription.sdp); session.media.forEach(function (media) { if (media.ssrcs != null && media.ssrcs.length > 0) { // TODO(gp) maybe exclude FID streams? media.ssrcs.forEach(function (ssrc) { if (ssrc.attribute !== 'cname') { return; } newssrcs.push({ 'ssrc': ssrc.id, 'type': media.type }); }); } else if(self.localStreamsSSRC && self.localStreamsSSRC[media.type]) { newssrcs.push({ 'ssrc': self.localStreamsSSRC[media.type], 'type': media.type }); } }); console.log('new ssrcs', newssrcs); // Bind us as local SSRCs owner if (newssrcs.length > 0) { for (var i = 1; i <= newssrcs.length; i ++) { var ssrc = newssrcs[i-1].ssrc; var myJid = self.connection.emuc.myroomjid; self.ssrcOwners[ssrc] = myJid; if (newssrcs[i-1].type === 'video'){ self.ssrcVideoTypes[ssrc] = self.getVideoType(); } } } } // an attempt to work around https://github.com/jitsi/jitmeet/issues/32 function sendKeyframe(pc) { console.log('sendkeyframe', pc.iceConnectionState); if (pc.iceConnectionState !== 'connected') return; // safe... var self = this; pc.setRemoteDescription( pc.remoteDescription, function () { pc.createAnswer( function (modifiedAnswer) { pc.setLocalDescription( modifiedAnswer, function () { // noop }, function (error) { console.log('triggerKeyframe setLocalDescription failed', error); eventEmitter.emit(XMPPEvents.SET_LOCAL_DESCRIPTION_ERROR); } ); }, function (error) { console.log('triggerKeyframe createAnswer failed', error); eventEmitter.emit(XMPPEvents.CREATE_ANSWER_ERROR); } ); }, function (error) { console.log('triggerKeyframe setRemoteDescription failed', error); eventEmitter.emit(XMPPEvents.SET_REMOTE_DESCRIPTION_ERROR); } ); } JingleSession.prototype.remoteStreamAdded = function (data, times) { var self = this; var thessrc; var streamId = APP.RTC.getStreamID(data.stream); // look up an associated JID for a stream id if (!streamId) { console.error("No stream ID for", data.stream); } else if (streamId && streamId.indexOf('mixedmslabel') === -1) { // look only at a=ssrc: and _not_ at a=ssrc-group: lines var ssrclines = SDPUtil.find_lines(this.peerconnection.remoteDescription.sdp, 'a=ssrc:'); ssrclines = ssrclines.filter(function (line) { // NOTE(gp) previously we filtered on the mslabel, but that property // is not always present. // return line.indexOf('mslabel:' + data.stream.label) !== -1; if (RTCBrowserType.isTemasysPluginUsed()) { return ((line.indexOf('mslabel:' + streamId) !== -1)); } else { return ((line.indexOf('msid:' + streamId) !== -1)); } }); if (ssrclines.length) { thessrc = ssrclines[0].substring(7).split(' ')[0]; if (!self.ssrcOwners[thessrc]) { console.error("No SSRC owner known for: " + thessrc); return; } data.peerjid = self.ssrcOwners[thessrc]; data.videoType = self.ssrcVideoTypes[thessrc] console.log('associated jid', self.ssrcOwners[thessrc], thessrc, data.videoType); } else { console.error("No SSRC lines for ", streamId); } } APP.RTC.createRemoteStream(data, this.sid, thessrc); var isVideo = data.stream.getVideoTracks().length > 0; // an attempt to work around https://github.com/jitsi/jitmeet/issues/32 if (isVideo && data.peerjid && this.peerjid === data.peerjid && data.stream.getVideoTracks().length === 0 && APP.RTC.localVideo.getTracks().length > 0) { window.setTimeout(function () { sendKeyframe(self.peerconnection); }, 3000); } } module.exports = JingleSession;