Today's article tells you a story and process of tracking down bugs. I have always believed that if something goes wrong, there must be demons, and so is the Bug in the program.
I hope that through this Bug troubleshooting story, we can not only learn a series of knowledge points, but also learn how to solve problems and how to do things more professionally. The way of solving problems and thinking are more important than simple technology.
Let's go!
The cause of the story
Not long after taking over the new project of the new team, when releasing a system, colleagues kindly reminded: when releasing the xx system, comment out a line of code in the test environment and release the comments when releasing it online.
Hearing this friendly reminder, I was surprised: what kind of black technology is this?! In my experience, there is no system that needs to be handled in this way. I am secretly determined to troubleshoot this problem.
Finally, I took time. I tossed about for more than half a day on Friday and didn't solve it. I was still thinking about it at the weekend, so I worked overtime to solve this problem.
Existence and operation of Bug
The project is based on JSP without front-end and back-end separation. A public head.jsp is introduced into the JSP page, which contains such a line of code and comments:
<!-- Solve Online HTTPS Browser circling problem,The test environment should comment out the following sentence --> <meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests" />
What colleagues kindly remind is the operations on the comments. The test environment is commented out (otherwise it can't be accessed), and the production environment needs to be released, otherwise it can't be accessed (turn around). According to the notes, it is probably used to solve HTTPS related problems.
So, what is the reason for this operation? Is there a simpler operation? We are just doing this. No one is looking for the root of the problem, and no one can give the answer. We can only look for it ourselves.
HTTP requests in HTTPS
Let's take a look at what META elements are used for.
The "content security policy" specified by HTTP equiv is "web page security policy", abbreviated as CSP, which is often used to prevent XSS attacks.
The usual way to use it is to define it in HTML through meta Tags:
<meta http-equiv="content-security-policy" content="strategy"> <meta http-equiv="content-security-policy-report-only" content="strategy">
Among them, various restriction policies related to security can be specified in content.
The upgrade secure requests used in the project is one of the restriction policies. Its function is to automatically replace all HTTP links loaded with external resources on the web page with HTTPS protocol.
Now I understand a little. It turns out that this line of code was originally written to forcibly convert HTTP requests into HTTPS requests.
But normally, as long as HTTP to HTTPS is configured in Nginx or SLB, such problems will not occur, and there is a corresponding configuration in the system.
So, I started another service experiment online and commented out this code. Some functions are really turning around. I'm not deceived!
Why are HTTP requests not allowed in HTTPS
Viewing the request in the browser, it is found that the circle is caused by the following error:
Mixed Content: The page at 'https://example.com' was loaded over HTTPS, but requested an insecure stylesheet 'http://example.com/xxx'. This request has been blocked; the content must be served over HTTPS.
Mixed Content is Mixed Content. The so-called Mixed Content usually occurs in the following situations: the initial HTML content is loaded through HTTPS, but other resources (such as css style, js, pictures, etc.) are loaded through unsafe HTTP requests. At this time, the same page uses both HTTP and HTTPS content, and the HTTP protocol will reduce the security of the whole page.
Therefore, modern browsers will warn against HTTP requests in HTTPS, block requests, and throw the above exception information.
Now, the cause of the problem is basically clear: HTTP requests appear in HTTPS requests.
So, there are several solutions:
- Scheme 1: add meta tags in HTML to force the conversion of HTTP requests into HTTPS requests. This is also the above use method, but the disadvantages of this method are also obvious. In the test environment without HTTPS, it needs to be commented out manually. Otherwise, it cannot be accessed normally.
- Scheme 2: convert HTTP requests into HTTPS requests through the configuration of Nginx or SLB.
- Scheme 3: the stupidest method is to find the problems of HTTP requests in the project and repair them one by one.
Preliminary transformation, slightly effective
The first scheme currently used obviously does not meet the requirements, while the second scheme has been configured, but some pages still do not work. So, are there any other options?
After a lot of investigation, it is found that the reason for the failure is that the redirect jump is widely used in the project.
@RequestMapping(value = "delete")
public String delete(RedirectAttributes redirectAttributes) {
//.. do something
addMessage(redirectAttributes, "delete xxx success");
return "redirect:" + Global.getAdminPath() + "/list";
}
The redirect method will redirect to the HTTP protocol in the HTTPS environment, resulting in inaccessibility.
It's too bad. No wonder the settings of HTTP to HTTPS have been configured, and some pages still don't work.
The root cause of this problem is the compatibility of Spring's ViewResolver with HTTP 1.0 protocol.
This problem can be solved by closing it. There are two specific transformation schemes.
Scheme 1: change redirect to the RedirectView class to implement:
modelAndView.setView(new RedirectView(Global.getAdminPath() + "/list", true, false));
The last parameter of RedirectView is set to false, which is to turn off the http10Compatible switch and not be compatible with the HTTP 1.0 protocol.
Scheme 2: configure the redirectHttp10Compatible property of Spring's ViewResolver. Through this scheme, global shutdown can be realized.
<bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="viewClass" value="org.springframework.web.servlet.view.JstlView" /> <property name="prefix" value="/" /> <property name="suffix" value=".jsp" /> <property name="redirectHttp10Compatible" value="false" /> </bean>
Because redirect is widely used in the project, the second scheme is adopted. After modification, it is found that most of the problems have been solved.
In order to prevent omission, I ordered more pages, and even missed fish!
Shiro interceptor again
The problem caused by redirection was solved. I thought everything was fine. As a result, similar problems occurred on the page redirected by Shiro. The reason is simple: the permission verification of some pages needs to go through Shiro, but Shiro intercepts the HTTPS request and converts it into an HTTP request during redirection.
So why doesn't it work if the view layer sets redirectHttp10Compatible to false?
After tracking the code in Shiro interceptor, it is found that Shiro sets redirectHttp10Compatible to true by default in the interceptor, which is another pit~
Looking at the source code, you can find that Shiro's login filter FormAuthenticationFilter calls the saveRequestAndRedirectToLogin method:
protected void saveRequestAndRedirectToLogin(ServletRequest request, ServletResponse response) throws IOException {
saveRequest(request);
redirectToLogin(request, response);
}
// Then call the redirectToLogin method
protected void redirectToLogin(ServletRequest request, ServletResponse response) throws IOException {
String loginUrl = getLoginUrl();
WebUtils.issueRedirect(request, response, loginUrl);
}
// Set through WebUtils.issueRedirect
public static void issueRedirect(ServletRequest request, ServletResponse response, String url) throws IOException {
issueRedirect(request, response, url, (Map)null, true, true);
}
// Overloaded method via WebUtils.issueRedirect
public static void issueRedirect(ServletRequest request, ServletResponse response, String url, Map queryParams, boolean contextRelative, boolean http10Compatible) throws IOException {
RedirectView view = new RedirectView(url, contextRelative, http10Compatible);
view.renderMergedOutputModel(queryParams, toHttp(request), toHttp(response));
}
Through the above code tracing, we can see that in the WebUtils issueRedirect method, we call two times issueRedirect, and the http10Compatible parameter value is true by default.
Find the root of the problem and solve it easily. Rewrite the FormAuthenticationFilter Interceptor:
public class CustomFormAuthenticationFilter extends FormAuthenticationFilter {
@Override
protected boolean onAccessDenied(ServletRequest request, ServletResponse response) throws Exception {
if (isLoginRequest(request, response)) {
if (isLoginSubmission(request, response)) {
return executeLogin(request, response);
} else {
return true;
}
} else {
saveRequestAndRedirectToLogin(request, response);
return false;
}
}
protected void saveRequestAndRedirectToLogin(ServletRequest request, ServletResponse response) throws IOException {
saveRequest(request);
redirectToLogin(request, response);
}
protected void redirectToLogin(ServletRequest request, ServletResponse response) throws IOException {
String loginUrl = getLoginUrl();
WebUtils.issueRedirect(request, response, loginUrl, null, true, false);
}
}
In the example, change the http10Compatible parameter in onAccessDenied that needs to call the WebUtils.issueRedirect method to false.
The above is just an example. In fact, it includes not only the success page, but also the failure page. You need to re implement the corresponding methods. Finally, configure a custom interceptor in shiroFilter.
<!-- Custom login filter--> <bean id="customFilter" class="com.senzhuang.shiro.CustomFormAuthenticationFilter" /> <bean id="shiroFilter" class="org.apache.shiro.spring.web.ShiroFilterFactoryBean"> <property name="securityManager" ref="securityManager" /> <property name="loginUrl" value="/login.html"></property> <property name="unauthorizedUrl" value="/refuse.html"></property> <property name="filters"> <map> <entry key="authc" value-ref="customFilter"/> </map> </property> </bean>
After the above transformation, the problem of HTTP request in HTTPS has been solved.
In order to prevent omission, I clicked some pages one by one and sent questions again! Hey, why do you owe me so much
LayUI pit
I thought that after solving the above problems, it was completely solved. I could have a barbecue to celebrate. As a result, a similar error was found in the front page. However, the error message comes from the path to the login page:
http://example.com/a/login
It's strange that you have successfully logged in. Why does the business operation page request the login page again? Moreover, the jump is still an HTTP request, not an HTTPS request.
Check the login request results:
After checking the relevant business codes, after the login is completed, there is no login request. Why do you request login again? Is it because access to some resources is limited, which leads to redirection to the login page?
So, check the "Initiator" called by HTML:
It turned out that the login operation of login was triggered when LayUI requested the corresponding layer.css resource.
The first thought is that Shiro did not release the interception of static resources, so the interception permission of layui was released in Shiro, but the problem already exists.
After troubleshooting again, it is found that the layer.css file is not actively introduced into the page, so the layer.css file is actively introduced, but the problem still exists.
I have no choice but to check layui.js to see why the request is made. At this time, I also notice that there is a word "undefined CSS" in the request path.
Friends who have used js know that undefined is the uninitialized default value of variables in js, similar to null in Java.
Search "css /" in layui.js, and you really find such a code:
return layui.link(o.dir + "css/" + e, t, n)
In contrast, the value of o.dir is "undefined", which becomes "undefined" when connected with the following css. However, this path does not exist, and the permission is not configured in Shiro. By default, it will go to the login interface. Here is an internal asynchronous redirect request, which will not be presented on the page. You can find it only by viewing the browser's error information.
Once the cause of the problem is found, the transformation is simple. Modify the link method parameter of layui:
// Comment out // return layui.link(o.dir + "css/" + e, t, n) // Change to return layui.link((o.dir ? o.dir:"/static/sc_layui/") +"css/"+e, t, n)
The basic idea of the transformation is: if o.dir has a value (the value in js is true), then use the value of o.dir; if o.dir is undefined, then use the specified default value.
Where "/ static/sc_layui /" is the path to store layui components in the project. Since layui.js may be compressed JS, you can find the corresponding code by searching "css /" or "layui.link".
Restart the project, clear the browser cache, and visit the page again. The problem has been completely solved.
You can eat kebabs at ease
It took another half day at the weekend and finally solved the problem. Now you can have a kebab to celebrate.
Finally, review the process and see what you can gain from it:
- There is a problem: the code needs to be changed manually in different environments (HTTP and HTTPS);
- Looking for problems: for security, HTTP requests are not allowed in HTTPS;
- Problem solving: close http10Compatible in two ways;
- Shiro problem: http10Compatible is turned off by default in Shiro, and Filter is rewritten to realize the closing operation;
- LayUI Bug repair: a bug in the LayUI code causes an http (login) request to be initiated. Repair this bug;
In this process, if you are just content with the status quo and "abide by the rules", it will not only take time and effort, but also do not know why you want to do so every time you go online.
But if you follow up like the author, you will learn a series of knowledge:
- CSP of HTTP request, upgrade secure requests configuration;
- Why can't an HTTP request be initiated in HTTPS;
- Configure http10Compatible in the Spring View parser;
- Disadvantages of view return in redirect mode;
- How to convert HTTP requests into HTTPS requests in Nginx;
- Mixed Content concept and error of HTTP request;
- Differences between HTTP 1.0, HTTP 1.1 and HTTP 2.0 protocols;
- Shiro interceptor custom Filter;
- Shiro interceptor filters the specified URL access;
- Shiro interceptor configuration and partial source code implementation;
- A bug in LayUI;
- Other technologies used or learned in troubleshooting the problem;
Have you learned these techniques? Have you learned the ideas and ways to solve problems? If there is a little content in this article that inspires you, I will not hesitate to share it, and you should not be stingy. Give me a praise.
About the blogger: the author of the technical book "inside of SpringBoot technology", loves to study technology and write technical dry goods articles.
The official account: "new horizon of procedures", the official account of bloggers, welcome the attention.
Technical exchange: please contact blogger wechat: zhuan2quan
