[AutoTinyPng] compress the picture from the programmer's point of view

Keywords: node.js

Dear, Hello, I'm "front-end Xiaoxin", ­čśç He has been engaged in front-end development and Android development for a long time, is keen on technology, and goes farther and farther on the road of programming

preface:

It's strange to say that there are some "pseudo programmers" in many technical communication groups. For example, in the dialogue below, the online image compression website wants to compress a large number of its own images. It's too troublesome to go to the group and ask what to do?

Solve this problem from the programmer's point of view:
  1. Fishing at work: one by one, one by one.
  2. Tyrant krypton gold method: simple programming through the open API of the website for batch processing. Of course, the more you process, you need to pay some fees.
  3. Show the technical method: it is suitable for reviewing your programming knowledge in a reasonable number and a rare opportunity, and can do a good job.
  4. other:...

Preparation before coding:

  1. We chose to show the technical method to do today's Demo. I also think it is a programmer's choice (I lost it to the artist...);
  2. The quality of a product also needs to be polished and optimized gradually. Tinypng is still a popular product among programmers. We still choose tinypng and use other professional tools to do our own things. It's beautiful!.

    Idea introduction:

  3. Recursively get the files in the local folder
  4. Filter file, format must be. Jpg. PNG, size less than 5MB. (folder recursion)
  5. Only one file is processed at a time (the limit of 20 can be bypassed)
  6. Process the returned data to get the remote optimized picture address
  7. Retrieve pictures and update local pictures
  8. A pure node implementation does not rely on any other code snippets

    Coding implementation:

    Only applicable to modules provided by Node:
    const fs = require("fs");
    const { Console } = require("console");
    const path = require("path");
    const https = require("https");
    const URL = require("url").URL;
    Common browser ID to prevent the same ID from being intercepted by the server:
    const USER_AGENT = [
      "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
      "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv,2.0.1) Gecko/20100101 Firefox/4.0.1",
      "Mozilla/5.0 (Windows NT 6.1; rv,2.0.1) Gecko/20100101 Firefox/4.0.1",
      "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11",
      "Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11",
      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11",
      "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 360SE)",
      "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; maxthon 2.0)",
      "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SE 2.X MetaSr 1.0; SE 2.X MetaSr 1.0; .NET CLR 2.0.50727; SE 2.X MetaSr 1.0)",
    ];
    Define log and output log to file:
    // Define log and support outputting log to file
    class Log {
      options = {
     flags: "a", // append mode
     encoding: "utf8", // utf8 encoding
      };
    
      logger = {};
    
      /**
    * Initialize Print Configuration
    */
      constructor() {
     this.logger = new Console({
       stdout: fs.createWriteStream("./log.tinypng.stdout.log", this.options),
       stderr: fs.createWriteStream("./log.tinypng.stderr.log", this.options),
     });
      }
    
      /**
    * log level
    * @param {*} message Output information
    */
      log(message) {
     if (message) {
       this.logger.log(message);
       console.log(message);
     }
      }
    
      /**
    * error level
    * @param {*} message Output err information
    */
      error(message) {
     if (message) {
       this.logger.error(message);
       console.error(message);
     }
      }
    }
    
    // Instantiate Log object
    const Tlog = new Log();
    Define TinyPng object:
    class TinyPng {
      // Configuration information: suffix format and maximum file size are limited by reception and cannot be adjusted
      config = {
     files: [],
     entryFolder: "./",
     deepLoop: false,
     extension: [".jpg", ".png"],
     max: 5200000, // 5MB == 5242848.754299136
     min: 100000, // 100KB
      };
    
      // Successful processing count
      successCount = 0;
      // Failed processing count
      failCount = 0;
    
      /**
    * TinyPng constructor 
    * @param {*} entry Entry file
    * @param {*} deep Recursive
    */
      constructor(entry, deep) {
     console.log(USER_AGENT[Math.floor(Math.random() * 10)]);
     if (entry != undefined) {
       this.config.entryFolder = entry;
     }
     if (deep != undefined) {
       this.config.deepLoop = deep;
     }
     // Filter the pending files that match the adjustment in the incoming entry directory
     this.fileFilter(this.config.entryFolder);
     Tlog.log(`Configuration of this execution script:`);
     Object.keys(this.config).forEach((key) => {
       if (key !== "files") {
         Tlog.log(`to configure ${key}: ${this.config[key]}`);
       }
     });
     Tlog.log(`Number of files waiting to be processed: ${this.config.files.length}`);
      }
    
      /**
    * Perform compression
    */
      compress() {
     Tlog.log("Start image compression,One moment please...");
     let asyncAll = [];
     if (this.config.files.length > 0) {
       this.config.files.forEach((img) => {
         asyncAll.push(this.fileUpload(img));
       });
       Promise.all(asyncAll)
         .then(() => {
           Tlog.log(
             `Processing completed: success: ${this.successCount}Zhang, Success rate ${
               this.successCount / this.config.files.length
             }`
           );
         })
         .catch((error) => {
           Tlog.error(error);
         });
     }
      }
    
      /**
    * Filter the pending folder to get the list of pending files
    * @param {*} folder Pending folder
    * @param {*} files List of pending files
    */
      fileFilter(folder) {
     // Read folder
     fs.readdirSync(folder).forEach((file) => {
       let fullFilePath = path.join(folder, file);
       // Read file information
       let fileStat = fs.statSync(fullFilePath);
       // Filter file security / size limit / suffix
       if (
         fileStat.size <= this.config.max &&
         fileStat.size >= this.config.min &&
         fileStat.isFile() &&
         this.config.extension.includes(path.extname(file))
       ) {
         this.config.files.push(fullFilePath);
       }
       // Deep recursive processing of folders is required
       else if (this.config.deepLoop && fileStat.isDirectory()) {
         this.fileFilter(fullFilePath);
       }
     });
      }
    
      /**
    * TinyPng Configuration generation method of remote compressed HTTPS request
    */
      getAjaxOptions() {
     return {
       method: "POST",
       hostname: "tinypng.com",
       path: "/web/shrink",
       headers: {
         rejectUnauthorized: false,
         "X-Forwarded-For": Array(4)
           .fill(1)
           .map(() => parseInt(Math.random() * 254 + 1))
           .join("."),
         "Postman-Token": Date.now(),
         "Cache-Control": "no-cache",
         "Content-Type": "application/x-www-form-urlencoded",
         "User-Agent": USER_AGENT[Math.floor(Math.random() * 10)],
       },
     };
      }
    
      /**
    * TinyPng Remote compression HTTPS request
    * @param {string} img Documents to be processed
    * @success {
    *              "input": { "size": 887, "type": "image/png" },
    *              "output": { "size": 785, "type": "image/png", "width": 81, "height": 81, "ratio": 0.885, "url": "https://tinypng.com/web/output/7aztz90nq5p9545zch8gjzqg5ubdatd6" }
    *           }
    * @error  {"error": "Bad request", "message" : "Request is invalid"}
    */
      fileUpload(imgPath) {
     return new Promise((resolve) => {
       let req = https.request(this.getAjaxOptions(), (res) => {
         res.on("data", async (buf) => {
           let obj = JSON.parse(buf.toString());
           if (obj.error) {
             Tlog.log(`Compression failed!\n Current file: ${imgPath} \n ${obj.message}`);
           } else {
             resolve(await this.fileUpdate(imgPath, obj));
           }
         });
       });
       req.write(fs.readFileSync(imgPath), "binary");
       req.on("error", (e) => {
         Tlog.log(`Request error! \n Current file: ${imgPath} \n, ${e}`);
       });
       req.end();
     }).catch((error) => {
       Tlog.log(error);
     });
      }
    
      // This method is called circularly to request picture data
      fileUpdate(entryImgPath, obj) {
     return new Promise((resolve) => {
       let options = new URL(obj.output.url);
       let req = https.request(options, (res) => {
         let body = "";
         res.setEncoding("binary");
         res.on("data", (data) => (body += data));
         res.on("end", () => {
           fs.writeFile(entryImgPath, body, "binary", (err) => {
             if (err) {
               Tlog.log(err);
             } else {
               this.successCount++;
               let message = `Compression succeeded : Optimized proportion: ${(
                 (1 - obj.output.ratio) *
                 100
               ).toFixed(2)}% ´╝îOriginal size: ${(obj.input.size / 1024).toFixed(
                 2
               )}KB ´╝îCompression size: ${(obj.output.size / 1024).toFixed(
                 2
               )}KB ´╝îFile: ${entryImgPath}`;
               Tlog.log(message);
               resolve(message);
             }
           });
         });
       });
       req.on("error", (e) => {
         Tlog.log(e);
       });
       req.end();
     }).catch((error) => {
       Tlog.log(error);
     });
      }
    }
    
    module.exports = TinyPng;
    
    Entry script:
    /**
     * Some image processing failed due to network reasons and technical limitations such as anti brushing of third-party interface
     */
    const TinyPng = require("./tinypng.compress.img");
    
    function getEntryPath() {
      let i = process.argv.findIndex((i) => i === "-p");
      if (process.argv[i + 1]) {
     return process.argv[i + 1];
      }
    }
    new TinyPng(getEntryPath(), true).compress();
    Perform the demo:

    Logging:

    Conclusion:

    Programmers still need to simplify the repetitive work. A few years ago, this script reduced the front-end project of more than 150 M to 20 ~ 30 megabytes. How can you have such a project? Is your project very large? To tell you the truth, it's caused by non-standard. You have to deal with the files accumulated over the years one by one. Do you think it's reliable? What if you just compressed a pile of big pictures submitted by other colleagues? Then you'd better change the script and add it to the plug-in at compile time. Perfect!

Welcome to my official account, "front-end little Xin", and push the original technical article for the first time.

Posted by keeps21 on Fri, 26 Nov 2021 19:08:35 -0800