Overview of Memcached
An Open Source High Performance Distributed Memory Object Cache System
All data is stored in memory
Supports data of any storage type
Speed up Web site access
Memcached Caching Mechanism
When a program writes a cache data request, the API interface of Memcached routes the KEY input routing algorithm module to a service in the cluster, and then the API interface communicates with the server to complete a distributed cache write The Key index is built on the API, and the value value value data exists in the following memcached
Memcached Distributed
To be implemented by a Memcached-dependent client
Multiple Memcached servers are independent
How distributed data is stored is determined by routing algorithms
Memcached routing algorithm
Remainder hash algorithm Hash operation with key to an integer, then hash algorithm, based on the remainder of the route.Not suitable for dynamic environments Consistency hash algorithm The corresponding key s are mapped to a closed loop through a hash algorithm according to the hash algorithm, and then the machine is mapped to the loop by using the hash algorithm like object storage. The clockwise calculation stores all the objects to the nearest machine.Suitable for use in dynamic changes
Memcached is a distributed memory object caching system developed by danga.com, the technical team that runs LiveJournal, to reduce database load and improve performance in dynamic systems.I'm sure a lot of people have used this. This article is intended to get a better understanding of this great open source software through the implementation of memcached and code analysis, and to further optimize it according to our needs.Finally, we will further understand how memcached is used by analyzing the BSM_Memcache extension.
What is Memcached
Before explaining this issue, we first need to be clear about what it is not.Many people use it as a storage medium like SharedMemory. Although memcached organizes data in the same way as Key=>Value, it is very different from local caches such as shared memory and APC.Memcached is distributed, that is, it is not local.It is based on network connections (and of course it can use localhost) to complete services, and is itself an application-independent program or daemon (Daemon mode). Memcached uses the libevent library to implement network connection services and can theoretically handle an infinite number of connections, but unlike Apache, it is more often oriented to stable, continuous connections, so its actual concurrency capabilities are limited.Conservatively, the maximum number of simultaneous connections for memcached is 200, which is adjustable depending on Linux threading capabilities.You can refer to the documentation for libevent.Memcached uses memory differently than APC does.APC is based on shared memory and MMAP. memcachd has its own memory allocation algorithm and management method. It has nothing to do with shared memory, and there is no limit on shared memory. Usually, each memcached process can manage 2GB of memory space. If more space is needed, it can increase the number of processes.
What occasion is Memcached suitable for
In many cases, memcached is abused, and of course there are complaints about it.I often see people posting on the forums, similar to "How to improve efficiency". The reply is "Use memcached". There is nothing about how, where and what to do.Memcached is not everything, nor is it suitable for all situations.
Memcached is a "distributed" memory object caching system, that is, applications that do not need to be "distributed", shared, or simply small enough to have only one server do not benefit from memcached, but rather slow down system efficiency because network connections also require resources, even for UNIX local connections.In my previous test data, memcached reads and writes locally tens of times slower than direct PHP memory arrays, while APC and shared memory are all similar to direct arrays.You can see that memcached is very inexpensive if it's just a local-level cache.
Memcached is often used as a database front-end cache.Because it costs much less SQL parsing, disk operations, etc. than a database, and it uses memory to manage data, it can provide better performance than direct-read databases. In large systems, access to the same data is frequent, and memcached can greatly reduce database pressure and improve system execution efficiency.In addition, memcached is often used as a storage medium for data sharing between servers, such as saving system single sign-on status data in SSO systems to be stored in memcached and shared by multiple applications.
It is important to note that memcached uses memory to manage data, so it is volatile. When the server restarts, or the memcached process aborts, data is lost, so memcached cannot be used to persist data.Many people mistakenly understand that the performance of memcached is very good, good enough to compare memory with hard disk. In fact, memcached does not get hundreds or thousands of read and write speeds when using memory. The real bottleneck is network connection. It is much lighter than using disk's database system, because it does not have too much overhead and direct read and write side.The memcached process itself does not consume much CPU resources.
Let's start with the memcached cluster
Experimental environment
4 192.168.136.238 Master Server
5 192.168.136.239 Slave Server
6 192.168.136.185 Client
Client Access Drift Address 192.168.136.188**
4 Host server installs memcached, libevent event library, mamgent proxy package
[root@localhost ~]# mkdir /abc [root@localhost ~]# mount.cifs //192.168.100.25/memcached/abc #mount Password for root@//192.168.100.25/memcached: [root@localhost ~]# cd /abc/ [root@localhost abc]# ls LAMP-php5.6 magent-0.5.tar.gz memcached-1.5.6.tar.gz libevent-2.1.8-stable.tar.gz memcache-2.2.7.tgz [root@localhost abc]# tar zxvf libevent-2.1.8-stable.tar.gz -C /opt #Event libraries, memcached depends on event Libraries [root@localhost abc]# tar zxvf memcached-1.5.6.tar.gz -C /opt/ #Server-side memcached [root@localhost abc]# yum install gcc gcc-c++ make -y #Install Environment Package [root@localhost abc]# mkdir /opt/magent [root@localhost abc]# tar zxvf magent-0.5.tar.gz -C /opt/magent/ [root@localhost opt]# cd libevent-2.1.8-stable/ [root@localhost libevent-2.1.8-stable]# ./configure --prefix=/usr [root@localhost libevent-2.1.8-stable]# make && make install #Compile Installation [root@localhost libevent-2.1.8-stable]# cd ../memcached-1.5.6/ [root@localhost memcached-1.5.6]# ./configure --with-libevent=/usr make make install
5 Install memcached,libevent event library from server
[root@localhost ~]# mkdir /abc [root@localhost ~]# mount.cifs //192.168.100.25/memcached /abc Password for root@//192.168.100.25/memcached: [root@localhost ~]# cd /abc/ [root@localhost abc]# ls LAMP-php5.6 magent-0.5.tar.gz memcached-1.5.6.tar.gz libevent-2.1.8-stable.tar.gz memcache-2.2.7.tgz [root@localhost abc]# tar zxvf libevent-2.1.8-stable.tar.gz -C /opt #Event libraries, memcached depends on event Libraries [root@localhost abc]# tar zxvf memcached-1.5.6.tar.gz -C /opt/ #Server-side memcached [root@localhost abc]# yum install gcc gcc-c++ make -y [root@localhost opt]# cd libevent-2.1.8-stable/ [root@localhost libevent-2.1.8-stable]# ./configure --prefix=/usr [root@localhost libevent-2.1.8-stable]# make && make install [root@localhost libevent-2.1.8-stable]# cd ../memcached-1.5.6/ [root@localhost memcached-1.5.6]# ./configure --with-libevent=/usr make make install
4 Master Server Configuration magent
[root@localhost memcached-1.5.6]# cd /opt/ [root@localhost opt]# ls libevent-2.1.8-stable magent memcached-1.5.6 rh [root@localhost opt]# cd magent/ [root@localhost magent]# vim ketama.h #Change two lines #ifndef SSIZE_MAX #define SSIZE_MAX 32767 [root@localhost magent]# vim Makefile #Specify makefile file, change line LIBS = -levent -lm make #Compile [root@localhost magent]# ls ketama.c ketama.h ketama.o magent magent.c magent.o Makefile [root@localhost magent]# yum install openssh-clients -y #Install scp remote synchronization package [root@localhost magent]# cp magent /usr/bin/ #Put the magent script in / usr/local so that the system can recognize it [root@localhost magent]# scp magent root@192.168.136.239:/usr/bin/ #Copy the mangent file from the server
Firewall closed by master and slave
[root@localhost bin]# systemctl stop firewalld.service [root@localhost bin]# setenforce 0
Both master and slave
[root@localhost bin]# yum install keepalived -y
4 Master Server Configuration Master-Slave Synchronization
[root@localhost magent]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
vrrp_script magent { #Write a function script
script "/opt/shell/magent.sh" ##Specify script location
interval 2 ##Detection script interval
}
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id MAGENT_HA #Host server id, two cannot be the same
}
vrrp_instance VI_1 {
state MASTER
interface ens33 #Host Server Network Card
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111 #Default Validation
}
track_script { #Tuning function name magent
magent
}
virtual_ipaddress {
192.168.136.188 #Drift Address for Client Access
}
} 5 Install openssh client from server
[root@localhost bin]# cd /etc/keepalived/ [root@localhost keepalived]# mv keepalived.conf keepalived.conf.bak #Change name [root@localhost keepalived]# yum install openssh-clients -y
4 Primary server uses scp to transfer keepalived files from server
[root@localhost magent]# cd /etc/keepalived/ [root@localhost keepalived]# scp keepalived.conf root@192.168.136.239:/etc/keepalived/
5 Configure master-slave synchronization from a slave server
[root@localhost keepalived]# ls
keepalived.conf keepalived.conf.bak
[root@localhost keepalived]# vim keepalived.conf
! Configuration File for keepalived
vrrp_script magent {
script "/opt/shell/magent.sh"
interval 2
}
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id MAGENT_HB #routed_id cannot be the same
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 52 #The slave virtual id cannot be the same as the master server
priority 90 #Priority lower than primary server
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
magent
}
virtual_ipaddress {
192.168.136.188
}
} 4 Master server configuration magent script
[root@localhost keepalived]# mkdir /opt/shell
[root@localhost keepalived]# cd /opt/shell/
[root@localhost shell]# vim magent.sh
#!/bin/bash
k=`ps -ef | grep keepalived | grep -v grep | wc -l` #Check the keepaliveed process if it is turned on
if [ $k -gt 0 ]; then # -n Number of connections-l specifies drift address, -p specifies port mapping to master-slave server address
magent -u root -n 51200 -l 192.168.136.188 -p 12000 -s 192.168.136.238:11211 -b 192.168.136.239:11211
else
pkill -9 magent
fi
[root@localhost shell]# chmod +x magent.sh
[root@localhost shell]# systemctl start keepalived.service
[root@localhost shell]# netstat -ntap | grep 12000
tcp 0 0 192.168.136.188:12000 0.0.0.0:* LISTEN 124720/magent 5 Same operation from server
[root@localhost keepalived]# mkdir /opt/shell
[root@localhost keepalived]# cd /opt/shell/
[root@localhost shell]# vim magent.sh
#!/bin/bash
k=`ps -ef | grep keepalived | grep -v grep | wc -l`
if [ $k -gt 0 ]; then
magent -u root -n 51200 -l 192.168.136.188 -p 12000 -s 192.168.136.238:11211 -b 192.168.136.239:11211
else
pkill -9 magent
fi
[root@localhost shell]# chmod +x magent.sh
[root@localhost shell]# systemctl start keepalived.service
[root@localhost shell]# netstat -ntap | grep 12000 #View magent port
tcp 0 0 192.168.136.188:12000 0.0.0.0:* LISTEN 11660/magent Open 4 Master Server memcached
[root@localhost shell]# memcached -m 512k -u root -d -l 192.168.136.238 -p 11211 #Start master, -m specifies space size [root@localhost shell]# netstat -ntap | grep 11211 tcp 0 0 192.168.136.238:11211 0.0.0.0:* LISTEN 44647/memcached
Open 5 from server mememecached
[root@localhost shell]# memcached -m 512k -u root -d -l 192.168.136.239 -p 11211 #Start from [root@localhost shell]# netstat -ntap | grep 11211 tcp 0 0 192.168.136.239:11211 0.0.0.0:* LISTEN 42654/memcached
Client authentication uses telnet to connect to memcached memory database
[root@localhost ~]# telnet 192.168.136.188 12000 Trying 192.168.136.188... Connected to 192.168.136.188. Escape character is '^]'. add username 0 0 7 #Let's write a key-value pair of data first 1234567 STORED
Go to 4 Master Server to see if the data is available
[root@localhost shell]# telnet 192.168.136.238 11211 Trying 192.168.136.238... Connected to 192.168.136.238. Escape character is '^]'. geer^H^H ERROR get username VALUE username 0 7 1234567 END
Verify that 5 data from the server is synchronized
[root@localhost shell]# telnet 192.168.136.239 11211 Trying 192.168.136.239... Connected to 192.168.136.239. Escape character is '^]'. get username VALUE username 0 7 1234567 END