Front End Performance and Exception Monitoring

Keywords: Javascript DNS network axios

More articles

Preface

These days, there has been a lot of excitement to learn about front-end monitoring, but after looking at a lot of data, I found that there are no detailed articles about front-end monitoring, they are all about general, on the contrary, there are many existing front-end monitoring tools.

In order to learn more about the technical principles of front-end monitoring, I have been consulting the relevant data these days.Now I'm going to write an article detailing front-end monitoring to summarize my research over the past few days (and so on).

//Front End Monitoring Process
 Data Collection-->Data Update-->Server-side Processing-->Database Storage-->Data Monitoring Visualization Platform

However, this article only talks about the two steps of data collection and data reporting in monitoring, and the follow-up process requires the reader to explore and explore by himself (which is also a pleasure).

data acquisition

Performance Data Acquisition

Let's first learn about the Web API window.performance .

The PerformanceInterface provides access to performance-related information on the current page as part of the High Resolution Time API, which also incorporates the Performance Timeline API, Navigation Timing API, User Timing API, and Resource Timing API.

The property timing of this API contains the start and end times of each phase of page loading.


To make it easier for you to understand the meaning of timing's attributes, I know that I found a friend who wrote an introduction to timing, which I would like to reproduce here.

timing: {
        // Timestamp at the end of a page unload on the same browser.If there is no previous page, this value will be the same as fetchStart.
    navigationStart: 1543806782096,

    // Timestamp of the last page unload event thrown.If there is no previous page, this value returns 0.
    unloadEventStart: 1543806782523,

    // Corresponding to unloadEventStart, the timestamp at the end of unload event processing.If there is no previous page, this value returns 0.
    unloadEventEnd: 1543806782523,

    // Timestamp at the beginning of the first HTTP redirection.This value returns 0 if there is no redirection, or if there is a different source in the redirection.
    redirectStart: 0,

    // The timestamp of the last HTTP redirection (that is, the time when the last bit of the HTTP response was received directly).
    // If there is no redirection or a different source in the redirection, this value returns 0. 
    redirectEnd: 0,

    // The browser is ready to use an HTTP request to fetch the timestamp of the document.This point in time occurs before any application cache is checked.
    fetchStart: 1543806782096,

    // UNIX timestamp at the beginning of DNS domain name query.
        //If a persistent connection is used, or this information is stored on a cache or local resource, this value will match fetchStart.
    domainLookupStart: 1543806782096,

    // DNS domain name query completion time.
    //If a local cache (i.e. no DNS query) or persistent connection is used, it is equal to the fetchStart value
    domainLookupEnd: 1543806782096,

    // Timestamp for end of HTTP (TCP) domain name query.
        //If a persistent connection is used, or this information is stored on a cache or local resource, this value will match fetchStart.
    connectStart: 1543806782099,

    // HTTP (TCP) returns the timestamp at which the connection between the browser and the server was established.
        // If a persistent connection is established, the return value is equal to the value of the fetchStart property.Connection establishment refers to the complete completion of all handshakes and authentication processes.
    connectEnd: 1543806782227,

    // HTTPS returns the timestamp of the browser's handshake when it begins a secure link with the server.Returns 0 if the current page does not require a secure connection.
    secureConnectionStart: 1543806782162,

    // Returns the timestamp of the browser when it makes an HTTP request to the server (or when it starts reading the local cache).
    requestStart: 1543806782241,

    // Returns the timestamp of the first byte that the browser receives from the server (or reads from the local cache).
        //If the transport layer fails after starting the request and the connection is restarted, this property will be counted as the corresponding start time for the new request.
    responseStart: 1543806782516,

    // Returns the last byte the browser received from the server (or read from the local cache, or read from local resources)
        //(If the HTTP connection was closed before then, the time stamp when it was closed is returned).
    responseEnd: 1543806782537,

    // The timestamp of the beginning of the parsing of the DOM structure of the current Web page (that is, when the Document.readyState property becomes "load" and the corresponding readystatechange event is triggered).
    domLoading: 1543806782573,

    // The timestamp of the end of parsing the current page DOM structure and the start of loading embedded resources when the Document.readyState property becomes interactive and the corresponding readystatechange event triggers.
    domInteractive: 1543806783203,

    // When the parser sends the DOMContentLoaded event, the timestamp when all scripts that need to be executed have been parsed.
    domContentLoadedEventStart: 1543806783203,

    // Timestamp when all scripts that need to be executed immediately have been executed, regardless of execution order.
    domContentLoadedEventEnd: 1543806783216,

    // The current document parsing is complete, that is, the timestamp when the Document.readyState becomes'complete'and the corresponding readystatechange is triggered
    domComplete: 1543806783796,

    // Timestamp when the load event was sent.If this event has not yet been sent, its value will be 0.
    loadEventStart: 1543806783796,

    // When the load event ends, the timestamp when the load event completes.If this event has not yet been sent or completed, its value will be 0.
    loadEventEnd: 1543806783802
}

From the above data, we can get some useful time

// Redirection time-consuming
redirect: timing.redirectEnd - timing.redirectStart,
// White Screen Time
whiteScreen: timing.responseStart - timing.navigationStart,
// DOM Rendering Time-consuming
dom: timing.domComplete - timing.domLoading,
// Page Loading Time-consuming
load: timing.loadEventEnd - timing.navigationStart,
// Page Uninstall Time-consuming
unload: timing.unloadEventEnd - timing.unloadEventStart,
// Request Time-consuming
request: timing.responseEnd - timing.requestStart,
// Get Performance Information Current Time
time: new Date().getTime(),

In a few moments, you can see how well the first screen of a page loads.

In addition, using the window.performance.getEntriesByType('resource') method, we can also get the load time of related resources (js, css, img...), which will return all the resources currently loaded on the page.

It generally includes the following types

  • sciprt
  • link
  • img
  • css
  • fetch
  • other
  • xmlhttprequest

We only need the following information

// Name of the resource
name: item.name,
// Resource Loading Time-consuming
duration: item.duration.toFixed(2),
// Resource Size
size: item.transferSize,
// Protocols used for resources
protocol: item.nextHopProtocol,

Now, write a few lines of code to collect the data.

// Collect performance information
const getPerformance = () => {
    if (!window.performance) return
    const timing = window.performance.timing
    const performance = {
        // Redirection time-consuming
        redirect: timing.redirectEnd - timing.redirectStart,
        // White Screen Time
        whiteScreen: timing.responseStart - timing.navigationStart,
        // DOM Rendering Time-consuming
        dom: timing.domComplete - timing.domLoading,
        // Page Loading Time-consuming
        load: timing.loadEventEnd - timing.navigationStart,
        // Page Uninstall Time-consuming
        unload: timing.unloadEventEnd - timing.unloadEventStart,
        // Request Time-consuming
        request: timing.responseEnd - timing.requestStart,
        // Get Performance Information Current Time
        time: new Date().getTime(),
    }

    return performance
}

// Get resource information
const getResources = () => {
    if (!window.performance) return
    const data = window.performance.getEntriesByType('resource')
    const resource = {
        xmlhttprequest: [],
        css: [],
        other: [],
        script: [],
        img: [],
        link: [],
        fetch: [],
        // Current time to get resource information
        time: new Date().getTime(),
    }

    data.forEach(item => {
        const arry = resource[item.initiatorType]
        arry && arry.push({
            // Name of the resource
            name: item.name,
            // Resource Loading Time-consuming
            duration: item.duration.toFixed(2),
            // Resource Size
            size: item.transferSize,
            // Protocols used for resources
            protocol: item.nextHopProtocol,
        })
    })

    return resource
}

Summary

By interpreting performance and resource information, we can see that there are several reasons for the slow page loading:

  1. Too many resources
  2. Network speed is too slow
  3. Too many DOM elements

In addition to the slow speed of the user network, we can not, there are two other reasons that can be solved. There are already many performance-optimized articles and books on the network, and we are interested in finding out the information on our own.

Anomaly data collection

There are three types of exceptions/errors you can catch by now, as you can see from your data searches over the past few days.

  1. Resource Load Error
  2. js execution error
  3. promise error

1 Use addEventListener('error', callback, true) to catch resource load failure errors during capture phase.
2 Use window.onerror to catch js errors.
3 Use addEventListener('unhandledrejection', callback) to catch promise errors, but there is no information such as the number of rows and columns where errors occur, only relevant error information can be thrown manually.

We can build an error array variable errors, when an error occurs, add information about the error to the array, and then report it uniformly at a certain stage. See the code for details.

// Capture resource load failure error js css img...
addEventListener('error', e => {
    const target = e.target
    if (target != window) {
        monitor.errors.push({
            type: target.localName,
            url: target.src || target.href,
            msg: (target.src || target.href) + ' is load error',
            // When the error occurred
            time: new Date().getTime(),
        })
    }
}, true)

// Listen for js errors
window.onerror = function(msg, url, row, col, error) {
    monitor.errors.push({
        type: 'javascript',
        row: row,
        col: col,
        msg: error && error.stack? error.stack : msg,
        url: url,
        // When the error occurred
        time: new Date().getTime(),
    })
}

// The disadvantage of listening for promise errors is that row count data is not available
addEventListener('unhandledrejection', e => {
    monitor.errors.push({
        type: 'promise',
        msg: (e.reason && e.reason.msg) || e.reason || '',
        // When the error occurred
        time: new Date().getTime(),
    })
})

Summary

By collecting errors, you can learn the type and number of site exceptions, so you can make adjustments to reduce them.
Complete code and DEMO will be released at the end of the article, so you can copy the code (HTML file) and test it locally.

Data Update

Performance data reporting

Performance data can be reported after the page is loaded, with minimal impact on page performance.

window.onload = () => {
    // Get performance and resource information when the browser is idle
    // https://developer.mozilla.org/zh-CN/docs/Web/API/Window/requestIdleCallback
    if (window.requestIdleCallback) {
        window.requestIdleCallback(() => {
            monitor.performance = getPerformance()
            monitor.resources = getResources()
        })
    } else {
        setTimeout(() => {
            monitor.performance = getPerformance()
            monitor.resources = getResources()
        }, 0)
    }
}

Of course, you can also set a timer to circulate the report.However, it is better to make a comparison and re-report each time to avoid duplicate reporting of the same data.

Exception data reporting

The code I provided in DEMO is to collect all exceptions using an errors array and report them uniformly at a certain stage (delayed reporting).
In fact, you can also report an error when it occurs (instant report).This avoids the problem of losing anomalous data when the report is not triggered after the anomaly has been collected and the user has turned off the web page.

// Listen for js errors
window.onerror = function(msg, url, row, col, error) {
    const data = {
        type: 'javascript',
        row: row,
        col: col,
        msg: error && error.stack? error.stack : msg,
        url: url,
        // When the error occurred
        time: new Date().getTime(),
    }
    
    // Instant Update
    axios.post({ url: 'xxx', data, })
}

extend

SPA

The window.performance API has a disadvantage that when SPA switches routes, the window.performance.timing data is not updated.
So we need to find another way to count how long the switch routes to load complete.
For example, with Vue, one possible way is to execute the vm.$nextTick function in the component's beforeCreate hook to get the full rendering time of the component when switching routes.

beforeCreate() {
    const time = new Date().getTime()
    this.$nextTick(() => {
        this.$store.commit('setPageLoadedTime', new Date().getTime() - time)
    })
}


In addition to performance and exception monitoring, we can actually do more.

User Information Collection

navigator

Use window.navigator to collect user's device information, operating system, browser information...

UV(Unique visitor)

Refers to the natural person who visits and browses this web page through the Internet.A computer client accessing your website is a visitor.The same client is calculated only once within 00:00-24:00.Multiple visits to the same visitor within a day only count one UV.
When a user visits a website, a random string + time date can be generated and saved locally.When a request occurs on a Web page (regenerated if it exceeds 24 hours that day), these parameters are passed to the back end, which uses the information to generate UV statistics reports.

PV(Page View)

That is, page views or clicks, each time a user visits each page in the site, a PV is recorded.The cumulative number of times a user visits the same page, which measures the number of pages visited by site users.

Page dwell time

Traditional Web Site
When a user enters page A, he or she picks up the time when he or she enters the page through a background request.After 10 minutes, the user enters page B, at which point the background can tell by the parameters carried by the interface that the user stayed on page A for 10 minutes.
SPA
Router can be used to get the user's dwell time. Take Vue for example, and use the router.beforeEach destroyed hook function to get the user's dwell time on the routing component.

Browse Depth

Using the document.documentElement.scrollTop property and the screen height, you can determine whether the user has finished browsing the site content.

Page Jump Source

The document.referrer property lets you know from which website the user jumped.

Summary

By analyzing the user data, we can know the user's browsing habits, hobbies and so on. It's horrible to think about, there is no privacy at all.

DEMO

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <link rel="stylesheet" href="test.css">
    <title>Document</title>
</head>
<body>
    <button class="btn1">Exception Test Button 1</button>
    <button class="btn2">Exception Test Button 2</button>
    <button class="btn3">Exception Test Button 3</button>
    <img src="https://avatars3.githubusercontent.com/u/22117876?s=460&v=4" alt="">
    <img src="test.png" alt="">
<script>
function monitorInit() {
    const monitor = {
        // Data upload address
        url: '',
        // Performance Information
        performance: {},
        // resource information
        resources: {},
        // error message
        errors: [],
        // User Information
        user: {
            // Screen Width
            screen: screen.width,
            // Screen Height
            height: screen.height,
            // Browser Platform
            platform: navigator.platform,
            // Browser User Agent Information
            userAgent: navigator.userAgent,
            // Language of browser user interface
            language: navigator.language,
        },
        // Add errors manually
        addError(error) {
            const obj = {}
            const { type, msg, url, row, col } = error
            if (type) obj.type = type
            if (msg) obj.msg = msg
            if (url) obj.url = url
            if (row) obj.row = row
            if (col) obj.col = col
            obj.time = new Date().getTime()
            monitor.errors.push(obj)
        },
        // Reset monitor object
        reset() {
            window.performance && window.performance.clearResourceTimings()
            monitor.performance = getPerformance()
            monitor.resources = getResources()
            monitor.errors = []
        },
        // Empty error information
        clearError() {
            monitor.errors = []
        },
        // Upload monitoring data
        upload() {
            // Custom Upload
            // axios.post({
            //     url: monitor.url,
            //     data: {
            //         performance,
            //         resources,
            //         errors,
            //         user,
            //     }
            // })
        },
        // Set data upload address
        setURL(url) {
            monitor.url = url
        },
    }

    // Get performance information
    const getPerformance = () => {
        if (!window.performance) return
        const timing = window.performance.timing
        const performance = {
            // Redirection time-consuming
            redirect: timing.redirectEnd - timing.redirectStart,
            // White Screen Time
            whiteScreen: timing.responseStart - timing.navigationStart,
            // DOM Rendering Time-consuming
            dom: timing.domComplete - timing.domLoading,
            // Page Loading Time-consuming
            load: timing.loadEventEnd - timing.navigationStart,
            // Page Uninstall Time-consuming
            unload: timing.unloadEventEnd - timing.unloadEventStart,
            // Request Time-consuming
            request: timing.responseEnd - timing.requestStart,
            // Get Performance Information Current Time
            time: new Date().getTime(),
        }

        return performance
    }

    // Get resource information
    const getResources = () => {
        if (!window.performance) return
        const data = window.performance.getEntriesByType('resource')
        const resource = {
            xmlhttprequest: [],
            css: [],
            other: [],
            script: [],
            img: [],
            link: [],
            fetch: [],
            // Current time to get resource information
            time: new Date().getTime(),
        }

        data.forEach(item => {
            const arry = resource[item.initiatorType]
            arry && arry.push({
                // Name of the resource
                name: item.name,
                // Resource Loading Time-consuming
                duration: item.duration.toFixed(2),
                // Resource Size
                size: item.transferSize,
                // Protocols used for resources
                protocol: item.nextHopProtocol,
            })
        })

        return resource
    }

    window.onload = () => {
        // Get performance and resource information at browser idle time https://developer.mozilla.org/zh-CN/docs/Web/API/Window/requestIdleCallback
        if (window.requestIdleCallback) {
            window.requestIdleCallback(() => {
                monitor.performance = getPerformance()
                monitor.resources = getResources()
                console.log('Page Performance Information')
                console.log(monitor.performance)
                console.log('Page Resource Information')
                console.log(monitor.resources)
            })
        } else {
            setTimeout(() => {
                monitor.performance = getPerformance()
                monitor.resources = getResources()
                console.log('Page Performance Information')
                console.log(monitor.performance)
                console.log('Page Resource Information')
                console.log(monitor.resources)
            }, 0)
        }
    }

    // Capture resource load failure error js css img...
    addEventListener('error', e => {
        const target = e.target
        if (target != window) {
            monitor.errors.push({
                type: target.localName,
                url: target.src || target.href,
                msg: (target.src || target.href) + ' is load error',
                // When the error occurred
                time: new Date().getTime(),
            })

            console.log('All error messages')
            console.log(monitor.errors)
        }
    }, true)

    // Listen for js errors
    window.onerror = function(msg, url, row, col, error) {
        monitor.errors.push({
            type: 'javascript', // Error Type
            row: row, // Number of lines of code in case of errors
            col: col, // Number of code columns in case of errors
            msg: error && error.stack? error.stack : msg, // error message
            url: url, // wrong file
            time: new Date().getTime(), // When the error occurred
        })

        console.log('All error messages')
        console.log(monitor.errors)
    }

    // The disadvantage of listening for promise errors is that row count data is not available
    addEventListener('unhandledrejection', e => {
        monitor.errors.push({
            type: 'promise',
            msg: (e.reason && e.reason.msg) || e.reason || '',
            // When the error occurred
            time: new Date().getTime(),
        })

        console.log('All error messages')
        console.log(monitor.errors)
    })

    return monitor
}

const monitor = monitorInit()
</script>
<script src="192.168.10.15/test.js"></script>
<script>
document.querySelector('.btn1').onclick = () => {
    setTimeout(() => {
        console.log(button)
    }, 0)
}

document.querySelector('.btn2').onclick = () => {
    new Promise((resolve, reject) => {
        reject({
            msg: 'test.js promise is error'
        })
    })
}

document.querySelector('.btn3').onclick = () => {
    throw ('This is a manually thrown error')
}
</script>
</body>
</html>

Reference material

https://fex.baidu.com/blog/20...
https://github.com/wangweiang...

Posted by talltorp on Fri, 15 Nov 2019 21:07:58 -0800

Hot Keywords

©2024 Programmer Group