On Date or Time Stamp Types in Distributed Systems
Recently, using Inceptor to query fields of type Date or Timestamp in TDH, we have encountered many problems with unstable or incorrect return values. There are two main problems:
- Dates of some special Date types are extra hours and seconds, and they are all "01:00:00"
- Date/Timestamp type fields are stored in stored procedures and the output is erroneous and unstable
The following will be explained one by one.
Open the DST, and the JVM will set the DST forward one hour from zero on the start day.
After many times of narrowing down, only a few more dates in the entire data set will be locked out in seconds, and all of them will be one hour more. Such as:
...
1988-04-10 --> 1988-04-10 01:00:00
1991-04-14 --> 1991-04-14 01:00:00
...After searching for relevant information, DST was found. Daylight Saving Time).
After the zdump check, it is true that the operating system has turned on the DST:
hadooptest4: # zdump -v /etc/localtime | grep 1988
/etc/localtime Sat Apr 9 15:59:59 1988 UT = Sat Apr 9 23:59:59 1988 CST isdst=0 gmtoff=28800
/etc/localtime Sat Apr 9 16:00:00 1988 UT = Sun Apr 10 01:00:00 1988 CDT isdst=1 gmtoff=32400
/etc/localtime Sat Sep 10 14:59:59 1988 UT = Sat Sep 10 23:59:59 1988 CDT isdst=1 gmtoff=32400
/etc/localtime Sat Sep 10 15:00:00 1988 UT = Sat Sep 10 23:00:00 1988 CST isdst=0 gmtoff=28800This indicates that when zero on April 10, 1988, due to the implementation of summer time, DST, the clock will automatically move forward for an hour, which leads to the error of Date type value with more time and seconds.
However, when setting up and closing DST, there is another common sense breaking argument: Etc/GMT-8 is equivalent to UTC+8 or GMT+8. Reference resources tz database
In the "Etc" area, zones west of GMT have a positive sign and those east have a negative sign in their name (e.g "Etc/GMT-14" is 14 hours ahead/east of GMT.)
So the final solution is to close all JVM programs, set the time zone without DST, and restart the program:
# cp /etc/localtime /etc/localtime.bk
# ln /usr/share/zoneinfo/Etc/GMT-8 -sf /etc/localtimeThe time zone settings of each node are inconsistent and the Date/Timestamp values are unstable.
In distributed systems, it is necessary to ensure the same configuration of each node as possible, otherwise it will easily lead to some strange problems. Time zone setting is one point, even if it is set up in different cities in China, there will be subtle differences. The problem arises from a very common date:
Under the condition of 20170918, a large number of 20170917 appeared.
And the same sentence of SQL, the return value is different, 20170917 has more or less.
Firstly, the reasons for DST are excluded.
- Summer time no longer applies in 2017
- Even with summer time, September 18 does not belong to the start date (one hour ahead of zero hours) or the end date (one hour back to zero hours).
After code checking, it is found that the time zones of each node are quite different:
import java.util.TimeZone;
System.out.println("TimeZone: " + TimeZone.getDefault());
As can be seen from the above figure, there is a 16-hour difference between nodes, which is also the problem left by the closure of DST with Etc/GMT+8 without prior knowledge of the characteristics of Etc.
It is noteworthy that after the unified modification to Etc/GMT+8 and ntp synchronization time, the time error was found, and then the PRC (UTC+8/China Standard Time Zone CST) was modified. However, after the above code checking, the problem of time zone inconsistency still occurred. It is temporarily impossible to explain this phenomenon.
But revising the system's / etc / local time and ensuring consistency of / etc/sysconfig/clock content can solve this problem.
Finally, the modification process is attached:
echo 'show current system time'
date
echo 'rm /etc/localtime'
rm -rf /etc/localtime
echo 'show current UTC time'
date
echo 'link localtime to Etc/GMT-8'
ln /usr/share/zoneinfo/Etc/GMT-8 -sf /etc/localtime
echo 'sync time from NTP server'
sntp -r 88.3.224.1
echo 'show current system time'
date
echo 'check timezone'
java -cp /root/test-date-1.0.0.jar io.transwarp.test.TestDate
echo 'adjust and unify /etc/sysconfig/clock'
echo 'set HWCLOCK to "--localtime"'
sed -i 's/HWCLOCK=.*/HWCLOCK="--localtime"/g' /etc/sysconfig/clock
echo 'set SYSTOHC to "yes"'
sed -i 's/SYSTOHC=.*/SYSTOHC="yes"/g' /etc/sysconfig/clock
echo 'set TIMEZONE and DEFAULT_TIMEZONE to "Asia/Beijing"'
sed -i 's/TIMEZONE=.*/TIMEZONE="Asia\/Beijing"/g' /etc/sysconfig/clock
echo 'set system time back to hardware clock'
hwclock -w
echo 'show hwclock and hwclock --localtime'
hwclock && hwclock --localtime
echo 'check timezone'
java -cp /root/test-date-1.0.0.jar io.transwarp.test.TestDateReference article: