brief introduction

Objective to help oneself understand webrtc to realize end-to-end communication

  # Usage flow
  git clone https://gitee.com/wjj0720/webrtc.git
  cd ./webRTC
  npm i
  npm run dev

  # Visit 127.0.0.1:3003/test-1.html to demonstrate h5 media stream capture
  # Visit 127.0.0.1:3003/local.html to demonstrate rtc local transmission
  # Visit 127.0.0.1:3003/p2p.html to demonstrate LAN end-to-end video screen    

what is WebRTC

  Webrtc (WEB real time communication) is an API that supports real-time voice and video conversation of web browser.
  Open source on June 1, 2011 and included in the W3C recommendation standard of the World Wide Web Alliance with the support of Google, Mozilla and Opera

  Gossip: current mainstream real-time streaming media implementation
  RTP (real time Transport Protocol) a protocol plus control based on UDP protocol

  HLS (HTTP live stream) HTTP based streaming media transport protocol implemented by Apple

  RTMP (Real Time Messaging Protocol) Adobe based on TCP

  WebRTC google based on RTP protocol

WebRTC composition

• getUserMedia is responsible for obtaining the user's local multimedia data
• RTCPeerConnection is responsible for establishing P2P connection and transmitting multimedia data.
• A signaling channel provided by RTCDataChannel realizes bidirectional communication

h5 get media stream

Objective: turn on the camera to show the media stream to the page

MediaDevices documentation

  navigator.mediaDevices.getUserMedia({
    video: true, // Camera
    audio: true // Microphone
  }).then(steam => {
    // srcObject of video label
    video.srcObject = stream
  }).catch(e => {
    console.log(e)
  })

RTCPeerConnection

The RTCPeerConnection api provides the implementation of the method of creating, linking, maintaining and monitoring the closed connection of WebRTC
RTCPeerConnection MDN

1. webRTC process

  Take a < = > b as an example to create a p2p connection
  
  A terminal:
    1. Create RTCPeerConnection instance: peerA
    2. Add your own local media stream (audio and video) to the instance, peerA.addStream
    3. Monitor the media stream peerA.onaddstream transmitted from the remote end
    4. Create [SDP offer] to start a new WebRTC connection to a remote (at this time, the remote is also called the candidate)) peer, peerA.createOffer 
    5. Pass the offer to the caller through [signaling server]
    6. After receiving the answer, go to the [stun] service to get your own IP, and send it to the call amplifier through the signaling service.

  B terminal:
    1. Create RTCPeerConnection peerB after receiving the notification from signaling service.
    2. You also need to add your own local media stream to the communication peer b.addstream.
    3. Monitor the media stream peerA.onaddstream transmitted from the remote end
    4. Also create [SDP offer] peerA.createAnswer
    5. Pass Answer to the caller through [signaling server]
    6. After receiving the IP address of the other party, go to the [stun] service to get your own IP address and deliver it to the other party.

    At this point, the p2p connection triggers the double sending of the onaddstream event.

1. Signaling service

     Signaling server:
       The system responsible for call establishment, supervision and teardown in webRTC
     Why:
       webRTC is a p2p connection. Before connecting, how to obtain the information of the other party and how to send their own information to the other party need signaling service.

2. SDP

     What is SDP?
       SDP is completely a session description format -- it does not belong to the transport protocol
       It only uses different appropriate transport protocols, including session Notification Protocol (SAP), session initiation protocol (SIP), real-time streaming protocol (RTSP), email of MIME extension protocol and Hypertext Transfer Protocol (HTTP).
       SDP is a text-based protocol, which has strong scalability, so it has a wide range of applications.
     
     SDP in WebRTC
       SDP does not support negotiation of session content or media encoding. In webrtc, SDP is used to describe media information (encoding and decoding information), and RTP is used to realize media negotiation.

3. stun

     1. What is STUN
       STUN (Session Traversal Utilities for NAT) is a network protocol, which allows clients located behind nat (or multiple NAT) to find out their own public network address, find out which type of NAT they are located behind and the Internet port that NAT is bound to for a local port. This information is used to create UDP traffic between two hosts that are behind the NAT router at the same time. This way of communicating directly through a route is called through a wall.
     
     2. What is NAT
       NAT (Network Address Translation) was proposed in 1994. When some hosts in the private network have been assigned local IP addresses, but now they want to communicate with hosts on the Internet, they install NAT software on the router. The router with NAT software is called NAT router, which can pass a global IP address. When all hosts using local address communicate with the outside world, this way of using a small number of public IP addresses to represent more private IP addresses will help alleviate the exhaustion of available IP address space.
   
     3.WebRTC through the wall
       At present, the commonly used NAT penetration methods for UDP connections are: STUN, TURN, ICE, uPnP, etc. Among them, due to the combination of STUN and TURN, the ICE mode is used by webrtc.
       Free address provided by google: https://webrtc.github.io/samples/src/content/peerconnection/trick-ice/

SHOW THE CODE

1. Front end

   <!DOCTYPE html>
     <html lang="zh">
     <head>
       <meta charset="UTF-8" />
       <meta name="viewport" content="width=device-width, initial-scale=1.0" />
       <meta http-equiv="X-UA-Compatible" content="ie=edge" />
       <title>End to end</title>
     </head>
     <body>
       <div class="page-container">
         <div class="message-box">
           <ul class="message-list"></ul>
           <div class="send-box">
             <textarea class="send-content"></textarea>
             <button class="sendbtn">Send out</button>
           </div>
         </div>
         <div class="user-box">
           <video id="local-video" autoplay class="local-video"></video>
           <video id="remote-video" autoplay class="remote-video"></video>
           <p class="title">Online users</p>
           <ul class="user-list"></ul>
         </div>
         <div class="mask">
           <div class="mask-content">
             <input class="myname" type="text" placeholder="Enter user name to join room">
             <button class="add-room">join</button>
           </div>
         </div>
         <div class="video-box">
   
         </div>
       </div>
       <script src="/js/jquery.js"></script>
       <script src="/js/socket.io.js"></script>
       <script>
         // Just wrap it up.
         class Chat {
           constructor({ calledHandle, host, socketPath, getCallReject } = {}) {
             this.host = host
             this.socketPath = socketPath
             this.socket = null
             this.calledHandle = calledHandle
             this.getCallReject = getCallReject
             this.peer = null
             this.localMedia = null
           }
           async init() {
             this.socket = await this.connentSocket()
             return this
           }
           async connentSocket() {
             if (this.socket) return this.socket
             return new Promise((resolve, reject) => {
               let socket = io(this.host, { path: this.socketPath })
               socket.on("connect", () => {
                 console.log("Connection succeeded!")
                 resolve(socket)
               })
               socket.on("connect_error", e => {
                 console.log("Connection failed!")
                 throw e
                 reject()
               })
               // Call accepted
               socket.on('answer', ({ answer }) => {
                 this.peer && this.peer.setRemoteDescription(answer)
               })
               // Called event
               socket.on('called', callingInfo => {
                 this.called && this.called(callingInfo)
               })
               // Call denied
               socket.on('callRejected', () => {
                 this.getCallReject && this.getCallReject()
               })
   
               socket.on('iceCandidate', ({ iceCandidate }) => {
                 console.log('Remote add iceCandidate');
                 this.peer && this.peer.addIceCandidate(new RTCIceCandidate(iceCandidate))
               })
   
             })
           }
           addEvent(name, cb) {
             if (!this.socket) return
             this.socket.on(name, (data) => {
               cb.call(this, data)
             })
           }
           sendMessage(name, data) {
             if (!this.socket) return
             this.socket.emit(name, data)
           }
           // Get local media stream
           async  getLocalMedia() {
             let localMedia = await navigator.mediaDevices
               .getUserMedia({ video: { facingMode: "user" }, audio: true })
               .catch(e => {
                 console.log(e)
               })
             this.localMedia = localMedia
             return this
           }
           // Set up media streaming to video
           setMediaTo(eleId, media) {
             document.getElementById(eleId).srcObject = media
           }
           // Called response
           called(callingInfo) {
             this.calledHandle && this.calledHandle(callingInfo)
           }
           // Create RTC
           createLoacalPeer() {
             this.peer = new RTCPeerConnection()
             return this
           }
           // Add media stream to communication
           addTrack() {
             if (!this.peer || !this.localMedia) return
             //this.localMedia.getTracks().forEach(track => this.peer.addTrack(track, this.localMedia));
             this.peer.addStream(this.localMedia)
             return this
           }
           // Create SDP offer
           async createOffer(cb) {
             if (!this.peer) return
             let offer = await this.peer.createOffer({ OfferToReceiveAudio: true, OfferToReceiveVideo: true })
             this.peer.setLocalDescription(offer)
             cb && cb(offer)
             return this
           }
           async createAnswer(offer, cb) {
             if (!this.peer) return
             this.peer.setRemoteDescription(offer)
             let answer = await this.peer.createAnswer({ OfferToReceiveAudio: true, OfferToReceiveVideo: true })
             this.peer.setLocalDescription(answer)
             cb && cb(answer)
             return this
   
           }
           listenerAddStream(cb) {
             this.peer.addEventListener('addstream', event => {
               console.log('addstream Event triggering', event.stream);
               cb && cb(event.stream);
             })
             return this
           }
           // Monitor candidates
           listenerCandidateAdd(cb) {
             this.peer.addEventListener('icecandidate', event => {
               let iceCandidate = event.candidate;
               if (iceCandidate) {
                 console.log('Send out candidate To the far end');
                 cb && cb(iceCandidate);
               }
   
             })
             return this
           }
           // Test ice negotiation process
           listenerGatheringstatechange  () {
             this.peer.addEventListener('icegatheringstatechange', e => {
                console.log('ice In consultation: ', e.target.iceGatheringState);
             })
             return this
           }
           // Close RTC
           closeRTC() {
             // ....
           }
         }
       </script>
       <script>
         $(function () {
   
           let chat = new Chat({
             host: 'http://127.0.0.1:3003',
             socketPath: "/websocket",
             calledHandle: calledHandle,
             getCallReject: getCallReject
           })
   
           // Update user list view
           function updateUserList(list) {
             $(".user-list").html(list.reduce((temp, li) => {
               temp += `<li class="user-li">${li.name} <button data-calling=${li.calling} data-id=${li.id} class=${li.id === this.socket.id || li.calling ? 'cannot-call' : 'can-call'}> Conversation</button></li>`
               return temp
             }, ''))
           }
           // Update message li table view
           function updateMessageList(msg) {
             $('.message-list').append(`<li class=${msg.userId === this.socket.id ? 'left' : 'right'}>${msg.user}: ${msg.content}</li>`)
           }
   
           // Join the room
           $('.add-room').on('click', async () => {
             let name = $('.myname').val()
             if (!name) return
             $('.mask').fadeOut()
             await chat.init()
             // User join event
             chat.addEvent('updateUserList', updateUserList)
             // Message update event
             chat.addEvent('updateMessageList', updateMessageList)
   
             chat.sendMessage('addUser', { name })
   
           })
           // send message
           $('.sendbtn').on('click', () => {
             let sendContent = $('.send-content').val()
             if (!sendContent) return
             $('.send-content').val('')
             chat.sendMessage('sendMessage', { content: sendContent })
           })
   
           // Visual screen
           $('.user-list').on('click', '.can-call', async function () {
             // Callee information
             let calledParty = $(this).data()
             if (calledParty.calling) return console.log('The other party is on the phone');
   
             // Initial local video
             $('.local-video').fadeIn()
             await chat.getLocalMedia()
             chat.setMediaTo('local-video', chat.localMedia)
   
             chat.createLoacalPeer()
               .listenerGatheringstatechange()
               .addTrack()
               .listenerAddStream(function (stream) {
   
                 $('.remote-video').fadeIn()
                 chat.setMediaTo('remote-video', stream)
   
               })
               .listenerCandidateAdd(function (iceCandidate) {
   
                 chat.sendMessage('iceCandidate', { iceCandidate, id: calledParty.id })
   
               })
               .createOffer(function (offer) {
   
                 chat.sendMessage('offer', { offer, ...calledParty })
   
               })
           })
   
           //call is rejected
           function getCallReject() {
             chat.closeRTC()
             $('.local-video').fadeIn()
             console.log('Call denied');
           }
   
           // called
           async function calledHandle(callingInfo) {
             if (!confirm(`Whether to accept ${callingInfo.name}Video call for`)) {
               chat.sendMessage('rejectCall', callingInfo.id)
               return
             }
   
             $('.local-video').fadeIn()
             await chat.getLocalMedia()
             chat.setMediaTo('local-video', chat.localMedia)
   
             chat.createLoacalPeer()
               .listenerGatheringstatechange()
               .addTrack()
               .listenerCandidateAdd(function (iceCandidate) {
   
                 chat.sendMessage('iceCandidate', { iceCandidate, id: callingInfo.id })
   
               })
               .listenerAddStream(function (stream) {
   
                 $('.remote-video').fadeIn()
                 chat.setMediaTo('remote-video', stream)
   
               })
               .createAnswer(callingInfo.offer, function (answer) {
   
                 chat.sendMessage('answer', { answer, id: callingInfo.id })
   
               })
   
   
           }
   
         })
       </script>
     </body>
     </html>

2. back-end

     const SocketIO = require('socket.io')
     const socketIO = new SocketIO({
       path: '/websocket'
     })
   
     let userRoom = {
       list: [],
       add(user) {
         this.list.push(user)
         return this
       },
       del(id) {
         this.list = this.list.filter(u => u.id !== id)
         return this
       },
       sendAllUser(name, data) {
         this.list.forEach(({ id }) => {
           console.log('>>>>>', id)
           socketIO.to(id).emit(name, data)
         })
         return this
       },
       sendTo(id) {
         return (eventName, data) => {
           socketIO.to(id).emit(eventName, data)
         }
       },
       findName(id) {
         return this.list.find(u => u.id === id).name
       }
     }
   
     socketIO.on('connection', function(socket) {
       console.log('Join join.', socket.id)
   
       socket.on('addUser', function(data) {
         console.log(data.name, 'Join the room')
         let user = {
           id: socket.id,
           name: data.name,
           calling: false
         }
         userRoom.add(user).sendAllUser('updateUserList', userRoom.list)
       })
   
       socket.on('sendMessage', ({ content }) => {
         console.log('Forward message:', content)
         userRoom.sendAllUser('updateMessageList', { userId: socket.id, content, user: userRoom.findName(socket.id) })
       })
   
       socket.on('iceCandidate', ({ id, iceCandidate }) => {
         console.log('Forward channel')
         userRoom.sendTo(id)('iceCandidate', { iceCandidate, id: socket.id })
       })
   
       socket.on('offer', ({id, offer}) => {
         console.log('Forward offer')
         userRoom.sendTo(id)('called', { offer, id: socket.id, name: userRoom.findName(socket.id)})
       })
   
       socket.on('answer', ({id, answer}) => {
         console.log('Receive video');
         userRoom.sendTo(id)('answer', {answer})
       })
   
       socket.on('rejectCall', id => {
         console.log('Forward rejected video')
         userRoom.sendTo(id)('callRejected')
       })
   
       socket.on('disconnect', () => {
         // Disconnect delete
         console.log('Connection disconnect', socket.id)
         userRoom.del(socket.id).sendAllUser('updateUserList', userRoom.list)
       })
     })
   
     module.exports = socketIO
   
   
     // www.js that's not the point
     const http = require('http')
     const app = require('../app')
     const socketIO = require('../socket.js')
     const server = http.createServer(app.callback())
     socketIO.attach(server)
     server.listen(3003, () => {
       console.log('server start on 127.0.0.1:3003')
     })  

Set up STUN/TURN

Because I didn't have the money to buy a server. I didn't try.

coturn It is said that it is very convenient to build STUN/TURN service.

  # Compile
  cd coturn
  ./configure --prefix=/usr/local/coturn
  sudo make -j 4 && make install

  # To configure
  listening-port=3478        #Specify the port to listen on
  external-ip=39.105.185.198 #Specify the public IP address of the virtual machine
  user=aaaaaa:bbbbbb         #User name and password to access the stun/turn service
  realm=stun.xxx.cn          #Domain name, this must be set
 
  #start-up
  cd /usr/local/coturn/bin
  turnserver -c ../etc/turnserver.conf

  trickle-ice https://Webrtc.github.io/samples/src/content/peerconnection/trick-ice enter stun/turn address, user and password as required 
  //The input information is: 
    STUN or TURN URI The value is: turn:stun.xxx.cn
    //User name: aaaaaa
    //Password: bbbbbb

STUN parameter passing

  let ice = {"iceServers": [
    {"url": "stun:stun.l.google.com:19302"},  // Password free
    // TURN generally needs to be defined by itself
    {
      'url': 'turn:192.158.29.39:3478?transport=udp',
      'credential': 'JZEOEt2V3Qb0y27GRntt2u2PAYA=', // Password
      'username': '28224511:1379330808' // User name
    },
    {
      'url': 'turn:192.158.29.39:3478?transport=tcp',
      'credential': 'JZEOEt2V3Qb0y27GRntt2u2PAYA=',
      'username': '28224511:1379330808'
    }
  ]}
  // Multiple iceServers addresses can be provided, but RTC selects one for negotiation


  // What is instantiated is that the given parameter RTC will obtain the IP address behind the local wall when appropriate.
  let pc = new RTCPeerConnection(ice);

  /*
    // It is said that these free addresses can be used.
    stun:stun1.l.google.com:19302
    stun:stun2.l.google.com:19302
    stun:stun3.l.google.com:19302
    stun:stun4.l.google.com:19302
    stun:23.21.150.121
    stun:stun01.sipphone.com
    stun:stun.ekiga.net
    stun:stun.fwdnet.net
    stun:stun.ideasip.com
    stun:stun.iptel.org
    stun:stun.rixtelecom.se
    stun:stun.schlund.de
    stun:stunserver.org
    stun:stun.softjoys.com
    stun:stun.voiparound.com
    stun:stun.voipbuster.com
    stun:stun.voipstunt.com
    stun:stun.voxgratia.org
    stun:stun.xten.com
  */