Step by step front-end monitoring system: how to record user behavior?

Keywords: Javascript less Database React

Summary: Reproduce BUG scenes quickly by recording user behavior.

Author: One footprint at a time, one pit at a time
Original: Set up front-end monitoring system (alternative) User behavior statistics and monitoring section (how to quickly locate online problems)

Fundebug Copyright shall be owned by the original author when authorized to reproduce.

Step by step, set up front-end monitoring system series blogs:

Background: There are many monitoring systems on the market, most of which are charged. For small front-end projects, it must be a pain point.Another major reason is that while functionality is generic, it may not necessarily meet our own needs, so it may be a good idea to be self-sufficient.

This is the second chapter of building the front-end monitoring system, mainly describes how to statistics js errors, follow me step by step, you can also build your own front-end monitoring system.

Online Demo currently running: Front End Monitoring System

The code and instructions are in this article: Introduction and Code of Monitoring System

If the deployment is really cumbersome, the Demo system can provide 7 days of monitoring and I will maintain it for a long time: One-Click Deployment

The frontline project has always been a black box to the frontend apes.Once a project is online, we don't know what users are doing inside our project, where they jump, or if they made a mistake.It's only when online users have problems and we can't reproduce them that we feel despair.No matter how hard it is, there will always be a problem waiting for you.So if we can turn an online project into a white box, and let us know what users are doing online, it won't be difficult to reproduce. Is that a good thing for front-end programmers?

What I want to write next is an important function, because it greatly improves my problem solving ability and has a great impact on my work.

So far, let's see what I've done:

PV/UV statistical report, js error report and analysis, interface statistical report, page screenshots statistical report.Then, by adding the "User Click Behavior Report" that we're going to write today, we can basically analyze what a user is doing on the page.

1. How to record the behavior of online users

The basic behaviors of online users include: visiting pages, clicking behavior, requesting interface behavior, js error behavior, which can basically clearly record all the behaviors of users online.Of course, there are also: resource loading behavior, page scrolling behavior, elements into the user's view and so on. These are more detailed behavior statistics, which may be improved in the future, but the above four behaviors can already fulfill our statistical needs.

Visit the page, we already have js error behavior, now see how to count click behavior and request interface behavior.

Click behavior

// User behavior log, inherited from log base class MonitorBaseInfo
  function BehaviorInfo(uploadType, behaviorType, className, placeholder, inputValue, tagName, innerText) {
    setCommonProperty.apply(this);
    this.uploadType = uploadType;
    this.behaviorType = behaviorType;
    this.className = utils.b64EncodeUnicode(className);
    this.placeholder = utils.b64EncodeUnicode(placeholder);
    this.inputValue = utils.b64EncodeUnicode(inputValue);
    this.tagName = tagName;
    this.innerText = utils.b64EncodeUnicode(encodeURIComponent(innerText));
  }

/**
   * User Behavior Record Monitoring
   * @param project Project Details
   */
  function recordBehavior(project) {
    // Behavior Recording Switch
    if (project && project.record && project.record == 1) {
      // Before recording behavior, check if the url record changes
      checkUrlChange();
      // Record user click behavior data on elements
      document.onclick = function (e) {
        var className = "";
        var placeholder = "";
        var inputValue = "";
        var tagName = e.target.tagName;
        var innerText = "";
        if (e.target.tagName != "svg" && e.target.tagName != "use") {
          className = e.target.className;
          placeholder = e.target.placeholder || "";
          inputValue = e.target.value || "";
          innerText = e.target.innerText.replace(/\s*/g, "");
          // If the click is too long, intercept the upload
          if (innerText.length > 200) innerText = innerText.substring(0, 100) + "... ..." + innerText.substring(innerText.length - 99, innerText.length - 1);
          innerText = innerText.replace(/\s/g, '');
        }
        var behaviorInfo = new BehaviorInfo(ELE_BEHAVIOR, "click", className, placeholder, inputValue, tagName, innerText);
        behaviorInfo.handleLogInfo(ELE_BEHAVIOR, behaviorInfo);
      }
    }
  };

Let's first look at the code for click behavior. It's actually very simple. It rewrites the document's onclick method, and then saves the attributes, contents, etc. of the corresponding elements. However, we have spent so much effort to save so many logs, just to record the user's click behavior.Too much waste.Therefore, this click behavior statistic will be added to future persistence analysis, which will enable the function of logging without burying point, and make our monitoring system more powerful and rich.Retention analysis will refer to GrowingIo Interested in learning about it.

We need to record the element's className, tagName, innerText, etc. We need enough content to determine which button the user clicked on.This is a less intelligent approach and will be improved in the future when writing about persistent analysis functions, but it is now enough to meet our requirements.

Request Interface Behavior

// Interface request log, inherited from log base class MonitorBaseInfo
  function HttpLogInfo(uploadType, url, status, statusText, statusResult, currentTime) {
    setCommonProperty.apply(this);
    this.uploadType = uploadType;
    this.httpUrl = utils.b64EncodeUnicode(url);
    this.status = status;
    this.statusText = statusText;
    this.statusResult = statusResult;
    this.happenTime = currentTime;
  }

/**
   * Page Interface Request Monitoring
   */
  function recordHttpLog() {
    // Listen for the status of ajax
    function ajaxEventTrigger(event) {
      var ajaxEvent = new CustomEvent(event, {
        detail: this
      });
      window.dispatchEvent(ajaxEvent);
    }
    var oldXHR = window.XMLHttpRequest;
    function newXHR() {
      var realXHR = new oldXHR();
      realXHR.addEventListener('abort', function () { ajaxEventTrigger.call(this, 'ajaxAbort'); }, false);
      realXHR.addEventListener('error', function () { ajaxEventTrigger.call(this, 'ajaxError'); }, false);
      realXHR.addEventListener('load', function () { ajaxEventTrigger.call(this, 'ajaxLoad'); }, false);
      realXHR.addEventListener('loadstart', function () { ajaxEventTrigger.call(this, 'ajaxLoadStart'); }, false);
      realXHR.addEventListener('progress', function () { ajaxEventTrigger.call(this, 'ajaxProgress'); }, false);
      realXHR.addEventListener('timeout', function () { ajaxEventTrigger.call(this, 'ajaxTimeout'); }, false);
      realXHR.addEventListener('loadend', function () { ajaxEventTrigger.call(this, 'ajaxLoadEnd'); }, false);
      realXHR.addEventListener('readystatechange', function() { ajaxEventTrigger.call(this, 'ajaxReadyStateChange'); }, false);
      return realXHR;
    }

    window.XMLHttpRequest = newXHR;
    window.addEventListener('ajaxLoadStart', function(e) {
      var currentTime = new Date().getTime()
      setTimeout(function () {
        var url = e.detail.responseURL;
        var status = e.detail.status;
        var statusText = e.detail.statusText;
        if (!url || url.indexOf(HTTP_UPLOAD_LOG_API) != -1) return;
        var httpLogInfo = new HttpLogInfo(HTTP_LOG, url, status, statusText, "Initiate Request", currentTime);
        httpLogInfo.handleLogInfo(HTTP_LOG, httpLogInfo);
      }, 2000)
    });
    window.addEventListener('ajaxLoadEnd', function(e) {
      var currentTime = new Date().getTime()
      var url = e.detail.responseURL;
      var status = e.detail.status;
      var statusText = e.detail.statusText;
      if (!url || url.indexOf(HTTP_UPLOAD_LOG_API) != -1) return;
      var httpLogInfo = new HttpLogInfo(HTTP_LOG, url, status, statusText, "Request Return", currentTime);
      httpLogInfo.handleLogInfo(HTTP_LOG, httpLogInfo);
    });

  }

Let's take a look at the code for interface behavior statistics. Originally, I wanted to take it out separately, but now I don't have so much time to develop its related functions, so I just wrote a simple version.

Statistics of interface behavior include: Initiating requests, receiving requests, receiving status, requesting duration, through the front-end interface statistics and analysis, we can observe the quality of online interfaces, but also make appropriate adjustments to the front-end logic, to achieve the best results of page loading.Database field definitions are all in place Analysis Background You can go directly to the project.

First, we'll listen to the Ajax requests on the page. As shown above, we've written a code to listen to the Ajax requests (I'm thanks picked up on the web), we'll listen to all the Ajax requests on the page, we'll do an atomic analysis of the entire Ajax request process, and we'll be able to listen to anything at any time during the request process.Parts, very useful.However, it is important that if your project uses fetch to request data, these listens will not work.Because fetch code is injected by the browser, it must be executed with monitoring code first, and then you can listen on ajax.So you need to rewrite the fetch code after you have written the Ajax listener so that it works.Okay, that's not the focus of this space, so let's get to that.

2. How to Query the Behavior of Online Users

Finally, we've successfully uploaded the remaining two behavior records, so what if we queried them out?Let's first look at the results I looked up on the page.

Because the screen is too small to display all the records, the record information includes: the name of the behavior, when the behavior occurred, the page where the behavior occurred, the error information, the error screenshot, and the user-defined time to upload the screenshot.

There are a few minor issues that need attention when it comes to this.

Because Js is used as a probe, it is difficult to ensure that each log entry inserts the user's userId
So we define a customerKey for each user to distinguish between, and if the user does not uninstall the app and clean up the app's cache, the customerKey will remain the same
When querying a user's behavior record, you need to first query out all of the user's customerKeys (there may be more than one), then query with customerKey to get accurate results.

3. How to analyze the behavior of online users

In fact, we have done so much and recorded so much for this purpose: to analyze behavior, to quickly locate problems.

So how do we position the problem? I can give an example:

JS error blocking behavior, we can see before and after the error behavior, can quickly and accurately locate the problem.

A complex link hop forwarding error occurred.Some errors occur when front-end pages undergo complex jumps and fall back, and even testers can't test for this problem because any behavior of users online can occur.Often all we know is that he made a mistake on the last page he stopped.In this way, we can reproduce the user's behavior and thus the BUG after checking the behavior log
Interface exception.Normally, the front-end interface will set a timeout, but then, the back-end interface investigation found that normal, and the front-end is not able to perform normally, this problem does not show the error phenomenon, and online feedback is not accurate, the front-end can only back the pot.Logging, on the other hand, allows you to record the time the request was sent and the time it was returned, so you can see if it has timed out.
Online users don't give abnormal feedback at all. All they can do is tell you what they see at the last glance.God knows what they went through before.The end result is that there is a problem with the front end, and then back the pot, haha.

In short, we know what users do on the page, so we don't have to worry about problems anymore, and we don't have to rush around when we meet them.

About Fundebug

Fundebug Focus on JavaScript, WeChat applets, WeChat games, Alipay applets, React Native, Node.js, and real-time BUG monitoring for Java online applications.Since the official launch of November 11, 2016, Fundebug has handled a cumulative billion + error events. Payment customers include sunshine insurance, walnut programming, litchi FM, Palm 1 to 1, Weimai, Youth League and many other brand enterprises.Welcome to the free trial!

Posted by chocopi on Fri, 02 Aug 2019 20:19:09 -0700

Programmer Group