Node static resource server

Static resource server is classified from server function. As the name implies, it only deals with static resource requests of browser
Static resources here refer to html, js, css, pictures, audio and video... Other documents

Create server

   through the native http module, start a service to process file requests.

const http = require('http')
const fs = require('fs')
const url = require('url')

const server = http.createServer((req, res) => {
  const { pathname } = url.parse(req.url, true)
  fs.readFile(`static${pathname}`, (err, data) => { // Suppose the static resources are stored in the static directory
    if (err) {
      res.writeHeader(404)
      res.write('Not Found')
    } else {
      res.write(data)
    }
    res.end()
  })
})

server.listen(8080)

In fact, it is a very time-consuming process to read the file and send it to the browser through fs.readFile, because when the file is large, it will be sent to the browser only after all the files have been read to memory.

                        .

                        .

Static resource server in the form of stream

   use the createStream method provided by the native fs module to process the reading and sending of static resources in the form of streams, so as to improve the performance of the server.

const http = require('http')
const fs = require('fs')
const url = require('url')

const server = http.createServer((req, res) => {
  const { pathname } = url.parse(req.url, true)
  const stream = fs.createReadStream(`static${pathname}`)
  stream.pipe(res)

  stream.on('error', error => {
    res.writeHeader(404)
    res.write('Not Found')
    res.end()
  })
})

server.listen(8080)

But there is a big problem in this way, that is, the transmitted files are not compressed, so the online servers will waste a lot of bandwidth, that is, the cost.

Static resource server in compressed form

   through the native zlib module, the read stream file is compressed and sent to the browser.

const http = require('http')
const fs = require('fs')
const url = require('url')
const zlib = require('zlib')

const server = http.createServer((req, res) => {
  const gzip = zlib.createGzip()
  const { pathname } = url.parse(req.url, true)
  const stream = fs.createReadStream(`static${pathname}`)

  res.setHeader('content-encoding', 'gzip')
  stream.pipe(gzip).pipe(res)

  stream.on('error', error => {
    res.writeHeader(404)
    res.write('Not Found')
    res.end()
  })
})

server.listen(8080)

File size comparison

Response header comparison

   you can roughly calculate how much space is saved before and after compression:

(262 - 77.8)/262 = 0.703
 Therefore, it can be seen that the compressed function saves about 70% of the loan cost, which is really unexpected!!!

Static resource server with cache policy

                             .

Server cache policy

   in order to solve the above-mentioned problem of static resource re acquisition without update, it can be solved by setting different cache strategies, usually including strong cache and negotiation cache.

So what is the negotiation cache?
  1. When the resource is requested for the first time, the server will add the last modified field to the response header (case insensitive) to indicate the last update time of the resource;
  2. When the browser requests the resource again, the if modified since field will be automatically set in the request header, and the value is the last modified value returned by the server last time;
  3. When the server obtains if modified since in the request header, it will determine whether the last update time of the resource is the same:
      If it is the same, it will return to 304 browser to use the cache directly;
      If they are not the same, the last modified value will be set;

Take negotiation caching as an example. Let's see that node server implements caching strategy.

const http = require('http')
const fs = require('fs')
const url = require('url')
const zlib = require('zlib')

const server = http.createServer((req, res) => {
  const { pathname } = url.parse(req.url, true)
  console.log(pathname)

  fs.stat(`static${pathname}`, (err, stat) => { // fs.stat can get the basic information of the file
    if (err) {
      send404(res)
    } else {
      const time = stat.mtime.toUTCString()
      const if_modified_since = req.headers['if-modified-since'] // Headers in headers must be lowercase

      if (if_modified_since) { // Compare last update time
        const client_time = Math.floor(new Date(if_modified_since).getTime()/1000)
        const server_time = Math.floor(new Date(time).getTime()/1000)
        if (server_time <= client_time) {
          send304(res)
        } else {
          sendData(pathname, time, res)
        }
      } else {
        sendData(pathname, time, res)
      }
    }
  })
})

server.listen(8080)

const send404 = (res) => {
  res.writeHeader(404)
  res.write('Not Found')
  res.end()
}

const send304 = (res) => {
  res.writeHeader(304)
  res.write('Not Modified')
  res.end()
}

const sendData = (pathname, time, res) => {
  const stream = fs.createReadStream(`static${pathname}`)
  res.on('error', error => {
    send404(response)
  })

  res.setHeader('Last-Modified', time)
  stream.pipe(res)
}

Front and back request headers with cache policy

Static resource server for multiprocessing requests

                            . For an online server, this is certainly not possible, so how to solve it?
The answer is to process requests in a multiprocess manner.
The cluster module of    NodeJS can create subprocesses that share server ports. We all know that the NodeJS instance is running in a single thread. We can make full use of the multi-core system to enable a group of NodeJS processes to handle load tasks.

const http = require('http')
const os = require('os')
const cluster = require('cluster')

if (cluster.isMaster) {
  const cpu_counts = os.cpus().length
  for(let i=0; i<cpu_counts; i++) {
    cluster.fork() // Parent process create child process
  }
} else { // Subprocesses are dedicated to processing requests
  const server = http.createServer((req, res) => {
    // Processing requests...
  })
  server.listen(8080) // You can listen to the same port between parent and child processes
}

To sum up, we can build a Web Node static resource server by ourselves, which can save bandwidth and traffic, and has good performance and is not easy to shut down.

Welcome to visit Personal blog