Catalog
· Pseudo-asynchronous IO programming
· Selector
· Contrast of Four IO Programming and Reasons for Choosing Netty
· Problem Description and Solution
· Development of HTTP Protocol
· Development of WebSocket Protocol
· Private protocol stack development
Linux Network IO Model
File descriptor
1. The Linux kernel treats all external devices as files.
2. The read-write operation of a file calls the system command provided by the kernel and returns a file descripter (fd, file descriptor).
3. Read and write a socket will also have a corresponding descriptor, called socketfd (socket descriptor).
Blocking IO Model
1. The most commonly used IO model.
2. Default IO model.
3. Take socket interface as an example to illustrate the blocking IO model.
Non-blocking IO Model
1. General rotation training checks whether the kernel data is ready.
2. If the kernel data is not ready, an EWOULDBLOCK error is returned directly.
IO Multiplexing Model
1. Linux provides select/poll, in which a process passes one or more FDS to a select or poll system call, blocking the select operation, so that the select/poll can help the process detect whether multiple FDS are ready at the same time.
2. select/poll has some problems such as limited number of fd support and linear rotation training, so epoll based on event-driven mode should be used instead (call back function immediately when fd is ready).
Signal Driven IO Model
The process first calls sigaction to execute a non-blocking signal processing function, and the process continues to run. When the data is ready, a SIGIO signal is generated for the process, notifying the process to call recvfrom to read the data.
Asynchronous IO Model
1. The process tells the kernel to start an operation, and then notifies the process after the whole operation is completed by the kernel.
2. Differentiation from signal-driven IO model: signal-driven IO model only notifies the data ready; asynchronous IO model notifies the operation completed.
BIO programming
1. An independent Acceptor thread is responsible for monitoring the client connection. After receiving the connection, a new thread is created for each client to process the link. After processing, the thread is returned to the client through the output stream, and the thread is destroyed.
2. Problem: The relationship between the number of threads on the server side and the number of concurrent visits on the client side is 1:1. When the concurrent visits of clients become more and more large, the system will suffer from thread stack overflow and failure to create new threads, which will eventually lead to process downtime or deadlock.
Pseudo-asynchronous IO programming
1. When the new client accesses, the client Socket is encapsulated as a Task (realizing Runnable interface) and delivered to the thread pool for processing.
2. Benefits: Since the size of thread pool queue and the maximum number of threads can be set, resource usage is controllable, and the increase in the number of concurrent clients will not lead to resource exhaustion and downtime.
3. Question: The synchronous blocking model is still used in the underlying communication, which can not fundamentally solve the problem of long-time blocking threads when the response message is slow or the network transmission is slow.
NIO programming
Buffer and Channel
1. BIO is stream-oriented, processing one byte at a time; NIO is block-oriented, processing data in the form of blocks.
2. BIO's java.io. * has been reimplemented using NIO.
3. Buffer buffers store data ready to be written or read. Usually it's a byte array, but it can also be an array of other types or not.
4. Buffer type:
(a) ByteBuffer (commonly used)
b) CharBuffer
c) ShortBuffer
d) IntBuffer
e) LongBuffer
f) FloatBuffer
g) DoubleBuffer
5. Channel channels are bidirectional and can read or write data through them. All data is processed by Buffer and never written directly to Channel.
6. Write an example file.
1 import java.io.FileOutputStream; 2 import java.io.IOException; 3 import java.io.UnsupportedEncodingException; 4 import java.nio.ByteBuffer; 5 import java.nio.channels.FileChannel; 6 import java.util.Random; 7 import java.util.UUID; 8 9 public class Test { 10 11 private static byte[] getRandomData() { 12 int randomLength = new Random().nextInt(100); 13 StringBuilder data = new StringBuilder(); 14 for (int index = 0; index < randomLength; index++) { 15 data.append(UUID.randomUUID().toString()); 16 } 17 return data.toString().getBytes(); 18 } 19 20 public static void main(String[] args) { 21 FileOutputStream fileOutputStream = null; 22 try { 23 fileOutputStream = new FileOutputStream("D:/test.txt"); 24 FileChannel fileChannel = fileOutputStream.getChannel(); 25 ByteBuffer byteBuffer = null; 26 for (int index = 0; index < 1000; index++) { 27 byte[] data = getRandomData(); 28 if (byteBuffer == null) { 29 byteBuffer = ByteBuffer.wrap(data); 30 } else if (data.length > byteBuffer.capacity()) { 31 if (byteBuffer.position() > 0) { 32 byteBuffer.flip(); 33 fileChannel.write(byteBuffer); 34 byteBuffer.clear(); 35 } 36 byteBuffer = ByteBuffer.wrap(data); 37 } else if (data.length > byteBuffer.remaining()) { 38 byteBuffer.flip(); 39 fileChannel.write(byteBuffer); 40 byteBuffer.clear(); 41 } 42 43 byteBuffer.put(data); 44 } 45 byteBuffer.flip(); 46 fileChannel.write(byteBuffer); 47 byteBuffer.clear(); 48 49 } catch (IOException e) { 50 e.printStackTrace(); 51 } finally { 52 if (fileOutputStream != null) { 53 try { 54 fileOutputStream.close(); 55 } catch (IOException e) { 56 e.printStackTrace(); 57 } 58 } 59 } 60 } 61 62 }
7. Read file examples.
1 import java.io.FileInputStream; 2 import java.io.IOException; 3 import java.nio.ByteBuffer; 4 import java.nio.channels.FileChannel; 5 6 public class Test { 7 8 public static void main(String[] args) { 9 FileInputStream fileInputStream = null; 10 try { 11 fileInputStream = new FileInputStream("D:/test.txt"); 12 FileChannel fileChannel = fileInputStream.getChannel(); 13 ByteBuffer byteBuffer = ByteBuffer.allocate(64); 14 while (fileChannel.read(byteBuffer) > 0) { 15 byteBuffer.flip(); 16 while (byteBuffer.hasRemaining()) { 17 System.out.print((char) byteBuffer.get()); 18 } 19 byteBuffer.clear(); 20 } 21 22 } catch (IOException e) { 23 e.printStackTrace(); 24 } finally { 25 if (fileInputStream != null) { 26 try { 27 fileInputStream.close(); 28 } catch (IOException e) { 29 e.printStackTrace(); 30 } 31 } 32 } 33 } 34 35 }
8. Examples of copying files.
1 import java.io.IOException; 2 import java.io.RandomAccessFile; 3 import java.nio.ByteBuffer; 4 import java.nio.channels.FileChannel; 5 6 public class Test { 7 8 public static void main(String[] args) { 9 RandomAccessFile sourceFile = null; 10 RandomAccessFile targetFile = null; 11 try { 12 sourceFile = new RandomAccessFile("D:/test.txt", "r"); 13 targetFile = new RandomAccessFile("D:/test.txt.bak", "rw"); 14 FileChannel sourceFileChannel = sourceFile.getChannel(); 15 FileChannel targetFileChannel = targetFile.getChannel(); 16 ByteBuffer byteBuffer = ByteBuffer.allocate(64); 17 while (sourceFileChannel.read(byteBuffer) > 0) { 18 byteBuffer.flip(); 19 targetFileChannel.write(byteBuffer); 20 byteBuffer.clear(); 21 } 22 23 } catch (IOException e) { 24 e.printStackTrace(); 25 } 26 } 27 28 }
Deep into Buffer
1. Buffer can be understood as an array, which describes the state through the following three values:
a) position: the position of the next element;
b) limit: The total number of elements that can be read or written, the position is always less than or equal to limit;
c) capacity: Buffer's maximum capacity, limit is always less than or equal to capacity.
2. Take reading and writing as examples to illustrate Buffer.
(a) Create an 8-byte ByteBuffer. position=0, limit=8, capacity=8.
b) Read 3 bytes. position=3, limit=8, capacity=8.
c) Read 2 bytes. position=5, limit=8, capacity=8.
d) flip(). position=0, limit=5, capacity=8.
e) Write 4 bytes. position=4, limit=5, capacity=8.
f) Write 1 byte. position=5, limit=5, capacity=8.
g) Execute clear(). position=0, limit=8, capacity=8.
3. Two ways to create ByteBuffer:
(a) Create a fixed-size Buffer.
ByteBuffer.allocate(capacity)
b) Wrap arrays and their contents into Buffer s.
byte array[] = new byte[1024]; ByteBuffer buffer = ByteBuffer.wrap(array);
Selector
1. Selector is a multiplexer in IO multiplexing model.
2. JDK uses epoll.
AIO programming
1. AIO, also known as NIO 2.0, is an asynchronous IO model.
2. At JDK 7, four new asynchronous Channel s were added under the java.nio.channels package.
a) AsynchronousSocketChannel
b) AsynchronousServerSocketChannel
c) AsynchronousFileChannel
d) AsynchronousDatagramChannel
3. Write files using Future: execute asynchronously, blocking Future.get() until the result is achieved.
1 import java.io.IOException; 2 import java.nio.ByteBuffer; 3 import java.nio.channels.AsynchronousFileChannel; 4 import java.nio.file.Path; 5 import java.nio.file.Paths; 6 import java.nio.file.StandardOpenOption; 7 import java.util.ArrayList; 8 import java.util.List; 9 import java.util.Random; 10 import java.util.UUID; 11 import java.util.concurrent.ExecutionException; 12 import java.util.concurrent.Future; 13 14 public class Test { 15 16 private static byte[] getRandomData() { 17 int randomLength = new Random().nextInt(100); 18 StringBuilder data = new StringBuilder(); 19 for (int index = 0; index < randomLength; index++) { 20 data.append(UUID.randomUUID().toString()); 21 } 22 return data.append('\n').toString().getBytes(); 23 } 24 25 public static void main (String [] args) { 26 Path file = Paths.get("D:/test.txt"); 27 AsynchronousFileChannel asynchronousFileChannel = null; 28 try { 29 asynchronousFileChannel = AsynchronousFileChannel.open(file, StandardOpenOption.WRITE); 30 List<Future<Integer>> futures = new ArrayList<>(); 31 for (int index = 0; index < 10; index++) { 32 ByteBuffer byteBuffer = ByteBuffer.wrap(getRandomData()); 33 Future<Integer> future = asynchronousFileChannel.write(byteBuffer, 0); 34 futures.add(future); 35 } 36 for (Future<Integer> future : futures) { 37 Integer length = null; 38 try { 39 length = future.get(); 40 } catch (InterruptedException | ExecutionException e) { 41 e.printStackTrace(); 42 } 43 System.out.println("Bytes written: " + length); 44 } 45 46 } catch (IOException e) { 47 e.printStackTrace(); 48 } finally { 49 if (asynchronousFileChannel != null) { 50 try { 51 asynchronousFileChannel.close(); 52 } catch (IOException e) { 53 e.printStackTrace(); 54 } 55 } 56 } 57 } 58 }
4. Write files using Completion Handler: asynchronous execution, callback Completion Handler. Note: In the example, since the main thread is not blocked, that is, whether the asynchronous task results in the end of the main thread or not, sometimes the result is not visible, sleep for 5 seconds.
1 import java.io.IOException; 2 import java.nio.ByteBuffer; 3 import java.nio.channels.AsynchronousFileChannel; 4 import java.nio.channels.CompletionHandler; 5 import java.nio.file.Path; 6 import java.nio.file.Paths; 7 import java.nio.file.StandardOpenOption; 8 import java.util.Random; 9 import java.util.UUID; 10 11 public class Test { 12 13 private static byte[] getRandomData() { 14 int randomLength = new Random().nextInt(100); 15 StringBuilder data = new StringBuilder(); 16 for (int index = 0; index < randomLength; index++) { 17 data.append(UUID.randomUUID().toString()); 18 } 19 return data.append('\n').toString().getBytes(); 20 } 21 22 public static void main (String [] args) { 23 Path file = Paths.get("D:/test.txt"); 24 AsynchronousFileChannel asynchronousFileChannel = null; 25 try { 26 asynchronousFileChannel = AsynchronousFileChannel.open(file, StandardOpenOption.WRITE); 27 CompletionHandler<Integer, Object> completionHandler = new CompletionHandler<Integer, Object>() { 28 @Override 29 public void completed(Integer result, Object attachment) { 30 System.out.println("Bytes written: " + result); 31 } 32 @Override 33 public void failed(Throwable exc, Object attachment) { 34 } 35 }; 36 for (int index = 0; index < 10; index ++) { 37 ByteBuffer byteBuffer = ByteBuffer.wrap(getRandomData()); 38 asynchronousFileChannel.write(byteBuffer, 0, null, completionHandler); 39 } 40 41 } catch (IOException e) { 42 e.printStackTrace(); 43 } finally { 44 if (asynchronousFileChannel != null) { 45 try { 46 asynchronousFileChannel.close(); 47 } catch (IOException e) { 48 e.printStackTrace(); 49 } 50 } 51 } 52 try { 53 Thread.sleep(5000); 54 } catch (InterruptedException e) { 55 e.printStackTrace(); 56 } 57 } 58 }
5. Use Future to read files: execute asynchronously, blocking Future.get() until the result is obtained.
1 import java.io.IOException; 2 import java.nio.ByteBuffer; 3 import java.nio.channels.AsynchronousFileChannel; 4 import java.nio.file.Path; 5 import java.nio.file.Paths; 6 import java.nio.file.StandardOpenOption; 7 import java.util.concurrent.ExecutionException; 8 import java.util.concurrent.Future; 9 10 public class Test { 11 12 public static void main (String [] args) { 13 Path file = Paths.get("D:/test.txt"); 14 AsynchronousFileChannel asynchronousFileChannel = null; 15 try { 16 asynchronousFileChannel = AsynchronousFileChannel.open(file, StandardOpenOption.READ); 17 ByteBuffer byteBuffer = ByteBuffer.allocate(64); 18 int position = 0; 19 int length = 0; 20 do { 21 Future<Integer> future = asynchronousFileChannel.read(byteBuffer, position); 22 length = future.get(); 23 if (length > 0) { 24 byteBuffer.flip(); 25 System.out.print(new String(byteBuffer.array())); 26 byteBuffer.clear(); 27 } 28 position += length; 29 } while (length > 0); 30 31 } catch (IOException e) { 32 e.printStackTrace(); 33 } catch (InterruptedException e) { 34 e.printStackTrace(); 35 } catch (ExecutionException e) { 36 e.printStackTrace(); 37 } finally { 38 if (asynchronousFileChannel != null) { 39 try { 40 asynchronousFileChannel.close(); 41 } catch (IOException e) { 42 e.printStackTrace(); 43 } 44 } 45 } 46 } 47 }
6. Read files using Completion Handler: asynchronous execution, callback Completion Handler. Note: In the example, since the main thread is not blocked, that is, whether the asynchronous task results in the end of the main thread or not, sometimes the result is not visible, sleep for 5 seconds.
1 import java.io.IOException; 2 import java.nio.ByteBuffer; 3 import java.nio.channels.AsynchronousFileChannel; 4 import java.nio.channels.CompletionHandler; 5 import java.nio.file.Path; 6 import java.nio.file.Paths; 7 import java.nio.file.StandardOpenOption; 8 9 public class Test { 10 11 public static void main (String [] args) { 12 Path file = Paths.get("D:/test.txt"); 13 AsynchronousFileChannel asynchronousFileChannel = null; 14 try { 15 asynchronousFileChannel = AsynchronousFileChannel.open(file, StandardOpenOption.READ); 16 // 10 Each asynchronous task reads 64 bytes of the file header and outputs them after 5 seconds. 17 CompletionHandler<Integer, ByteBuffer> completionHandler = new CompletionHandler<Integer, ByteBuffer>() { 18 @Override 19 public void completed(Integer result, ByteBuffer byteBuffer) { 20 byteBuffer.flip(); 21 System.out.print(new String(byteBuffer.array())); 22 byteBuffer.clear(); 23 } 24 @Override 25 public void failed(Throwable exc, ByteBuffer byteBuffer) { 26 } 27 }; 28 for (int index = 0; index < 10; index++) { 29 ByteBuffer byteBuffer = ByteBuffer.allocate(64); 30 asynchronousFileChannel.read(byteBuffer, byteBuffer.limit() * index, byteBuffer, completionHandler); 31 } 32 33 } catch (IOException e) { 34 e.printStackTrace(); 35 } finally { 36 if (asynchronousFileChannel != null) { 37 try { 38 asynchronousFileChannel.close(); 39 } catch (IOException e) { 40 e.printStackTrace(); 41 } 42 } 43 } 44 try { 45 Thread.sleep(5000); 46 } catch (InterruptedException e) { 47 e.printStackTrace(); 48 } 49 } 50 }
Contrast of Four IO Programming and Reasons for Choosing Netty
1. contrast.
2. Reasons for choosing NIO framework Netty instead of JDK NIO class library.
a) NIO class libraries and API s are complex.
b) Additional skills such as Java multithreading programming are required.
(c) The reliability is not high and the workload and difficulty are very great.
d) The notorious epoll Bug led to Selector Airship Training.
Introduction to Netty
Development and deployment
1. Development environment: Import "netty-all-x.y.z.jar" into CLASSPATH.
2. Packaged deployment: Because it is a non-Web application, it is enough to build a jar package deployment.
Hello World
1. Configure Maven's pom.xml file.
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>5.0.0.Alpha1</version>
</dependency>
2. Time Server
1 import io.netty.bootstrap.ServerBootstrap; 2 import io.netty.channel.ChannelFuture; 3 import io.netty.channel.ChannelInitializer; 4 import io.netty.channel.ChannelOption; 5 import io.netty.channel.EventLoopGroup; 6 import io.netty.channel.nio.NioEventLoopGroup; 7 import io.netty.channel.socket.SocketChannel; 8 import io.netty.channel.socket.nio.NioServerSocketChannel; 9 10 public class TimeServer { 11 12 public void bind(int port) throws Exception { 13 // The server NIO Thread group line 14 EventLoopGroup bossGroup = new NioEventLoopGroup(); 15 EventLoopGroup workerGroup = new NioEventLoopGroup(); 16 try { 17 ServerBootstrap serverBootstrap = new ServerBootstrap(); 18 serverBootstrap.group(bossGroup, workerGroup) 19 .channel(NioServerSocketChannel.class) 20 .option(ChannelOption.SO_BACKLOG, 1024) 21 .childHandler(new ChildChannelHandler()); 22 // Bind ports, wait for success synchronously 23 ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); 24 // Waiting for the server listening port to close 25 channelFuture.channel().closeFuture().sync(); 26 } finally { 27 // Exit gracefully, release thread pool resources 28 workerGroup.shutdownGracefully(); 29 bossGroup.shutdownGracefully(); 30 } 31 } 32 33 private class ChildChannelHandler extends ChannelInitializer<SocketChannel> { 34 35 @Override 36 protected void initChannel(SocketChannel socketChannel) throws Exception { 37 socketChannel.pipeline().addLast(new TimeServerHandler()); 38 } 39 40 } 41 42 public static void main(String[] args) throws Exception { 43 new TimeServer().bind(8080); 44 } 45 46 }
3. Time Server Handler
1 import java.util.Date; 2 3 import io.netty.buffer.ByteBuf; 4 import io.netty.buffer.Unpooled; 5 import io.netty.channel.ChannelHandlerAdapter; 6 import io.netty.channel.ChannelHandlerContext; 7 8 public class TimeServerHandler extends ChannelHandlerAdapter { 9 10 @Override 11 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { 12 ByteBuf reqBuf = (ByteBuf) msg; 13 byte[] req = new byte[reqBuf.readableBytes()]; 14 reqBuf.readBytes(req); 15 String reqString = new String(req, "UTF-8"); 16 String respString = "QUERY TIME ORDER".equalsIgnoreCase(reqString) ? new Date().toString() : "BAD ORDER"; 17 ByteBuf respBuf = Unpooled.copiedBuffer(respString.getBytes()); 18 ctx.write(respBuf); 19 } 20 21 @Override 22 public void channelReadComplete(ChannelHandlerContext ctx) throws Exception { 23 ctx.flush(); 24 } 25 26 @Override 27 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { 28 ctx.close(); 29 } 30 31 }
4. Time Client
1 import io.netty.bootstrap.Bootstrap; 2 import io.netty.channel.ChannelFuture; 3 import io.netty.channel.ChannelInitializer; 4 import io.netty.channel.ChannelOption; 5 import io.netty.channel.EventLoopGroup; 6 import io.netty.channel.nio.NioEventLoopGroup; 7 import io.netty.channel.socket.SocketChannel; 8 import io.netty.channel.socket.nio.NioSocketChannel; 9 10 public class TimeClient { 11 12 public void connect(String host, int port) throws Exception { 13 EventLoopGroup group = new NioEventLoopGroup(); 14 try { 15 // Client NIO Thread group 16 Bootstrap bootstrap = new Bootstrap(); 17 bootstrap.group(group).channel(NioSocketChannel.class) 18 .option(ChannelOption.TCP_NODELAY, true) 19 .handler(new ChildChannelHandler()); 20 // Initiate an asynchronous connection operation 21 ChannelFuture channelFuture = bootstrap.connect(host, port).sync(); 22 // Waiting for Client Link to Close 23 channelFuture.channel().closeFuture().sync(); 24 25 } finally { 26 // Exit gracefully, release NIO Thread group 27 group.shutdownGracefully(); 28 } 29 } 30 31 private class ChildChannelHandler extends ChannelInitializer<SocketChannel> { 32 33 @Override 34 protected void initChannel(SocketChannel socketChannel) throws Exception { 35 socketChannel.pipeline().addLast(new TimeClientHandler()); 36 } 37 38 } 39 40 public static void main(String[] args) throws Exception { 41 new TimeClient().connect("127.0.0.1", 8080); 42 } 43 44 }
5. Time Client Handler for Time Client Client
1 import io.netty.buffer.ByteBuf; 2 import io.netty.buffer.Unpooled; 3 import io.netty.channel.ChannelHandlerAdapter; 4 import io.netty.channel.ChannelHandlerContext; 5 6 public class TimeClientHandler extends ChannelHandlerAdapter { 7 8 private final ByteBuf reqBuf; 9 10 public TimeClientHandler() { 11 byte[] req = "QUERY TIME ORDER".getBytes(); 12 reqBuf = Unpooled.buffer(req.length); 13 reqBuf.writeBytes(req); 14 } 15 16 @Override 17 public void channelActive(ChannelHandlerContext ctx) throws Exception { 18 ctx.writeAndFlush(reqBuf); 19 } 20 21 @Override 22 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { 23 ByteBuf respBuf = (ByteBuf) msg; 24 byte[] resp = new byte[respBuf.readableBytes()]; 25 respBuf.readBytes(resp); 26 String respString = new String(resp, "UTF-8"); 27 System.out.println(respString); 28 } 29 30 @Override 31 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { 32 ctx.close(); 33 } 34 35 }
Packing/unpacking
Problems and Solutions
1. TCP is a "stream protocol" which is an unbounded string of data.
2. The underlying layer of TCP does not understand the specific meaning of the upper layer business data, it will divide the packets according to the actual situation of the TCP buffer. So in business, it is believed that a complete package may be sent by TCP unpacking, or it may encapsulate multiple packages.
Small packages are sent in large packages.
3. Solutions to the industry's mainstream agreements are summarized:
(a) Message length. For example, the size of each message is fixed at 200 bytes, and the space-time space is insufficient to fill in the blank space.
b) Increase the return line break at the end of the package for segmentation. For example, FTP protocol.
c) The message is divided into header and body. The header contains fields of the total length (or body length) of the message.
d) More complex application layer protocols.
4. Netty provides a variety of coders to solve the sticky/unpacking problem.
LineBasedFrameDecoder
1. Principle: Traverse through the readable bytes in ByteBuf and find " n" or " r n" when it ends.
2. Supports two encoding methods: carrying or not carrying terminators; Supports configuring the maximum length of a single line (throwing exceptions if no line breaks are found beyond the maximum length, while ignoring the abnormal stream read before).
3. StringDecoder function: Convert the received object into a string, and then continue to call the subsequent Andler.
4. Optimized time server using LineBasedFrameDecoder.
a) Time Server
1 import io.netty.bootstrap.ServerBootstrap; 2 import io.netty.channel.ChannelFuture; 3 import io.netty.channel.ChannelInitializer; 4 import io.netty.channel.ChannelOption; 5 import io.netty.channel.EventLoopGroup; 6 import io.netty.channel.nio.NioEventLoopGroup; 7 import io.netty.channel.socket.SocketChannel; 8 import io.netty.channel.socket.nio.NioServerSocketChannel; 9 import io.netty.handler.codec.LineBasedFrameDecoder; 10 import io.netty.handler.codec.string.StringDecoder; 11 12 public class TimeServer { 13 14 public void bind(int port) throws Exception { 15 // The server NIO Thread group line 16 EventLoopGroup bossGroup = new NioEventLoopGroup(); 17 EventLoopGroup workerGroup = new NioEventLoopGroup(); 18 try { 19 ServerBootstrap serverBootstrap = new ServerBootstrap(); 20 serverBootstrap.group(bossGroup, workerGroup) 21 .channel(NioServerSocketChannel.class) 22 .option(ChannelOption.SO_BACKLOG, 1024) 23 .childHandler(new ChildChannelHandler()); 24 // Bind ports, wait for success synchronously 25 ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); 26 // Waiting for the server listening port to close 27 channelFuture.channel().closeFuture().sync(); 28 } finally { 29 // Exit gracefully, release thread pool resources 30 workerGroup.shutdownGracefully(); 31 bossGroup.shutdownGracefully(); 32 } 33 } 34 35 private class ChildChannelHandler extends ChannelInitializer<SocketChannel> { 36 37 @Override 38 protected void initChannel(SocketChannel socketChannel) throws Exception { 39 socketChannel.pipeline().addLast(new LineBasedFrameDecoder(1024)); 40 socketChannel.pipeline().addLast(new StringDecoder()); 41 socketChannel.pipeline().addLast(new TimeServerHandler()); 42 } 43 44 } 45 46 public static void main(String[] args) throws Exception { 47 new TimeServer().bind(8080); 48 } 49 50 }
b) Time Server Handler
1 import java.util.Date; 2 3 import io.netty.buffer.ByteBuf; 4 import io.netty.buffer.Unpooled; 5 import io.netty.channel.ChannelHandlerAdapter; 6 import io.netty.channel.ChannelHandlerContext; 7 8 public class TimeServerHandler extends ChannelHandlerAdapter { 9 10 @Override 11 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { 12 String reqString = (String) msg; 13 String respString = "QUERY TIME ORDER".equalsIgnoreCase(reqString) ? new Date().toString() : "BAD ORDER"; 14 respString += "\n"; 15 ByteBuf respBuf = Unpooled.copiedBuffer(respString.getBytes()); 16 ctx.write(respBuf); 17 } 18 19 @Override 20 public void channelReadComplete(ChannelHandlerContext ctx) throws Exception { 21 ctx.flush(); 22 } 23 24 @Override 25 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { 26 ctx.close(); 27 } 28 29 }
c) Time Client
1 import io.netty.bootstrap.Bootstrap; 2 import io.netty.channel.ChannelFuture; 3 import io.netty.channel.ChannelInitializer; 4 import io.netty.channel.ChannelOption; 5 import io.netty.channel.EventLoopGroup; 6 import io.netty.channel.nio.NioEventLoopGroup; 7 import io.netty.channel.socket.SocketChannel; 8 import io.netty.channel.socket.nio.NioSocketChannel; 9 import io.netty.handler.codec.LineBasedFrameDecoder; 10 import io.netty.handler.codec.string.StringDecoder; 11 12 public class TimeClient { 13 14 public void connect(String host, int port) throws Exception { 15 EventLoopGroup group = new NioEventLoopGroup(); 16 try { 17 // Client NIO Thread group 18 Bootstrap bootstrap = new Bootstrap(); 19 bootstrap.group(group).channel(NioSocketChannel.class) 20 .option(ChannelOption.TCP_NODELAY, true) 21 .handler(new ChildChannelHandler()); 22 // Initiate an asynchronous connection operation 23 ChannelFuture channelFuture = bootstrap.connect(host, port).sync(); 24 // Waiting for Client Link to Close 25 channelFuture.channel().closeFuture().sync(); 26 27 } finally { 28 // Exit gracefully, release NIO Thread group 29 group.shutdownGracefully(); 30 } 31 } 32 33 private class ChildChannelHandler extends ChannelInitializer<SocketChannel> { 34 35 @Override 36 protected void initChannel(SocketChannel socketChannel) throws Exception { 37 socketChannel.pipeline().addLast(new LineBasedFrameDecoder(1024)); 38 socketChannel.pipeline().addLast(new StringDecoder()); 39 socketChannel.pipeline().addLast(new TimeClientHandler()); 40 } 41 42 } 43 44 public static void main(String[] args) throws Exception { 45 new TimeClient().connect("127.0.0.1", 8080); 46 } 47 48 }
d) Time Client Handler for Time Client Client
1 import io.netty.buffer.ByteBuf; 2 import io.netty.buffer.Unpooled; 3 import io.netty.channel.ChannelHandlerAdapter; 4 import io.netty.channel.ChannelHandlerContext; 5 6 public class TimeClientHandler extends ChannelHandlerAdapter { 7 8 private final ByteBuf reqBuf; 9 10 public TimeClientHandler() { 11 byte[] req = "QUERY TIME ORDER\n".getBytes(); 12 reqBuf = Unpooled.buffer(req.length); 13 reqBuf.writeBytes(req); 14 } 15 16 @Override 17 public void channelActive(ChannelHandlerContext ctx) throws Exception { 18 ctx.writeAndFlush(reqBuf); 19 } 20 21 @Override 22 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { 23 String respString = (String) msg; 24 System.out.println(respString); 25 } 26 27 @Override 28 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { 29 ctx.close(); 30 } 31 32 }
DelimiterBasedFrameDecoder
1. Function: Message decoding with delimiter as end identifier of stream.
2. Time Server
1 import io.netty.bootstrap.ServerBootstrap; 2 import io.netty.buffer.ByteBuf; 3 import io.netty.buffer.Unpooled; 4 import io.netty.channel.ChannelFuture; 5 import io.netty.channel.ChannelInitializer; 6 import io.netty.channel.ChannelOption; 7 import io.netty.channel.EventLoopGroup; 8 import io.netty.channel.nio.NioEventLoopGroup; 9 import io.netty.channel.socket.SocketChannel; 10 import io.netty.channel.socket.nio.NioServerSocketChannel; 11 import io.netty.handler.codec.DelimiterBasedFrameDecoder; 12 import io.netty.handler.codec.string.StringDecoder; 13 14 public class TimeServer { 15 16 public void bind(int port) throws Exception { 17 // The server NIO Thread group line 18 EventLoopGroup bossGroup = new NioEventLoopGroup(); 19 EventLoopGroup workerGroup = new NioEventLoopGroup(); 20 try { 21 ServerBootstrap serverBootstrap = new ServerBootstrap(); 22 serverBootstrap.group(bossGroup, workerGroup) 23 .channel(NioServerSocketChannel.class) 24 .option(ChannelOption.SO_BACKLOG, 1024) 25 .childHandler(new ChildChannelHandler()); 26 // Bind ports, wait for success synchronously 27 ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); 28 // Waiting for the server listening port to close 29 channelFuture.channel().closeFuture().sync(); 30 } finally { 31 // Exit gracefully, release thread pool resources 32 workerGroup.shutdownGracefully(); 33 bossGroup.shutdownGracefully(); 34 } 35 } 36 37 private class ChildChannelHandler extends ChannelInitializer<SocketChannel> { 38 39 @Override 40 protected void initChannel(SocketChannel socketChannel) throws Exception { 41 ByteBuf delimiter = Unpooled.copiedBuffer("*&*".getBytes()); 42 socketChannel.pipeline().addLast(new DelimiterBasedFrameDecoder(1024, delimiter)); 43 socketChannel.pipeline().addLast(new StringDecoder()); 44 socketChannel.pipeline().addLast(new TimeServerHandler()); 45 } 46 47 } 48 49 public static void main(String[] args) throws Exception { 50 new TimeServer().bind(8080); 51 } 52 53 }
3. Time Server Handler
1 import java.util.Date; 2 3 import io.netty.buffer.ByteBuf; 4 import io.netty.buffer.Unpooled; 5 import io.netty.channel.ChannelHandlerAdapter; 6 import io.netty.channel.ChannelHandlerContext; 7 8 public class TimeServerHandler extends ChannelHandlerAdapter { 9 10 @Override 11 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { 12 String reqString = (String) msg; 13 String respString = "QUERY TIME ORDER".equalsIgnoreCase(reqString) ? new Date().toString() : "BAD ORDER"; 14 respString += "*&*"; 15 ByteBuf respBuf = Unpooled.copiedBuffer(respString.getBytes()); 16 ctx.write(respBuf); 17 } 18 19 @Override 20 public void channelReadComplete(ChannelHandlerContext ctx) throws Exception { 21 ctx.flush(); 22 } 23 24 @Override 25 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { 26 ctx.close(); 27 } 28 29 }
4. Time Client
1 import io.netty.bootstrap.Bootstrap; 2 import io.netty.buffer.ByteBuf; 3 import io.netty.buffer.Unpooled; 4 import io.netty.channel.ChannelFuture; 5 import io.netty.channel.ChannelInitializer; 6 import io.netty.channel.ChannelOption; 7 import io.netty.channel.EventLoopGroup; 8 import io.netty.channel.nio.NioEventLoopGroup; 9 import io.netty.channel.socket.SocketChannel; 10 import io.netty.channel.socket.nio.NioSocketChannel; 11 import io.netty.handler.codec.DelimiterBasedFrameDecoder; 12 import io.netty.handler.codec.string.StringDecoder; 13 14 public class TimeClient { 15 16 public void connect(String host, int port) throws Exception { 17 EventLoopGroup group = new NioEventLoopGroup(); 18 try { 19 // Client NIO Thread group 20 Bootstrap bootstrap = new Bootstrap(); 21 bootstrap.group(group).channel(NioSocketChannel.class) 22 .option(ChannelOption.TCP_NODELAY, true) 23 .handler(new ChildChannelHandler()); 24 // Initiate an asynchronous connection operation 25 ChannelFuture channelFuture = bootstrap.connect(host, port).sync(); 26 // Waiting for Client Link to Close 27 channelFuture.channel().closeFuture().sync(); 28 29 } finally { 30 // Exit gracefully, release NIO Thread group 31 group.shutdownGracefully(); 32 } 33 } 34 35 private class ChildChannelHandler extends ChannelInitializer<SocketChannel> { 36 37 @Override 38 protected void initChannel(SocketChannel socketChannel) throws Exception { 39 ByteBuf delimiter = Unpooled.copiedBuffer("*&*".getBytes()); 40 socketChannel.pipeline().addLast(new DelimiterBasedFrameDecoder(1024, delimiter)); 41 socketChannel.pipeline().addLast(new StringDecoder()); 42 socketChannel.pipeline().addLast(new TimeClientHandler()); 43 } 44 45 } 46 47 public static void main(String[] args) throws Exception { 48 new TimeClient().connect("127.0.0.1", 8080); 49 } 50 51 }
5. Time Client Handler for Time Client Client
1 import io.netty.buffer.ByteBuf; 2 import io.netty.buffer.Unpooled; 3 import io.netty.channel.ChannelHandlerAdapter; 4 import io.netty.channel.ChannelHandlerContext; 5 6 public class TimeClientHandler extends ChannelHandlerAdapter { 7 8 private final ByteBuf reqBuf; 9 10 public TimeClientHandler() { 11 byte[] req = "QUERY TIME ORDER*&*".getBytes(); 12 reqBuf = Unpooled.buffer(req.length); 13 reqBuf.writeBytes(req); 14 } 15 16 @Override 17 public void channelActive(ChannelHandlerContext ctx) throws Exception { 18 ctx.writeAndFlush(reqBuf); 19 } 20 21 @Override 22 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { 23 String respString = (String) msg; 24 System.out.println(respString); 25 } 26 27 @Override 28 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { 29 ctx.close(); 30 } 31 32 }
FixedLengthFrameDecoder
1. Principle: No matter how many packets it receives at a time, it will decode according to the fixed length set. If it is a half-package message, it will cache the half-package message and wait for the next package to arrive to assemble the package until a complete package is read.
2. EchoServer
1 import io.netty.bootstrap.ServerBootstrap; 2 import io.netty.channel.ChannelFuture; 3 import io.netty.channel.ChannelInitializer; 4 import io.netty.channel.ChannelOption; 5 import io.netty.channel.EventLoopGroup; 6 import io.netty.channel.nio.NioEventLoopGroup; 7 import io.netty.channel.socket.SocketChannel; 8 import io.netty.channel.socket.nio.NioServerSocketChannel; 9 import io.netty.handler.codec.FixedLengthFrameDecoder; 10 import io.netty.handler.codec.string.StringDecoder; 11 12 public class EchoServer { 13 14 public void bind(int port) throws Exception { 15 // The server NIO Thread group line 16 EventLoopGroup bossGroup = new NioEventLoopGroup(); 17 EventLoopGroup workerGroup = new NioEventLoopGroup(); 18 try { 19 ServerBootstrap serverBootstrap = new ServerBootstrap(); 20 serverBootstrap.group(bossGroup, workerGroup) 21 .channel(NioServerSocketChannel.class) 22 .option(ChannelOption.SO_BACKLOG, 1024) 23 .childHandler(new ChildChannelHandler()); 24 // Bind ports, wait for success synchronously 25 ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); 26 // Waiting for the server listening port to close 27 channelFuture.channel().closeFuture().sync(); 28 } finally { 29 // Exit gracefully, release thread pool resources 30 workerGroup.shutdownGracefully(); 31 bossGroup.shutdownGracefully(); 32 } 33 } 34 35 private class ChildChannelHandler extends ChannelInitializer<SocketChannel> { 36 37 @Override 38 protected void initChannel(SocketChannel socketChannel) throws Exception { 39 socketChannel.pipeline().addLast(new FixedLengthFrameDecoder(20)); 40 socketChannel.pipeline().addLast(new StringDecoder()); 41 socketChannel.pipeline().addLast(new EchoServerHandler()); 42 } 43 44 } 45 46 public static void main(String[] args) throws Exception { 47 new EchoServer().bind(8080); 48 } 49 50 }
3. EchoServer Handler
1 import io.netty.channel.ChannelHandlerAdapter; 2 import io.netty.channel.ChannelHandlerContext; 3 4 public class EchoServerHandler extends ChannelHandlerAdapter { 5 6 @Override 7 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { 8 System.out.println(msg); 9 } 10 11 @Override 12 public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { 13 ctx.close(); 14 } 15 16 }
4. Using telnet command test, when the length reaches 20 characters, the server prints.
Java serialization problem
Problem Description and Solution
1. Unable to cross-language. Java serialization is a private protocol within the Java language, which is not supported by other languages.
2. Serialized bitstream is too large. The larger the byte array after encoding, the more space it takes for storage, the higher the hardware cost of storage, and the more bandwidth it takes for network transmission, which results in lower throughput of the system.
3. Serialization performance is too low. Coding and decoding take a long time.
4. Solution: Coding and decoding framework, such as Google Protobuf, Message Pack. It's not going to go into depth here.
Development of HTTP Protocol
Netty HTTP
1. Because of the universality of HTTP protocol, many heterogeneous systems use HTTP protocol for communication interaction, such as HTTP + XML or RESTful + JSON, which is very popular.
2. Compared with Web container, Netty has the advantages of developing HTTP: lightweight; security.
3. Take the file server as an example. As for HTTP + XML, we don't expand in depth here.
File Server
1. File server HttpFileServer
1 import io.netty.bootstrap.ServerBootstrap; 2 import io.netty.channel.ChannelFuture; 3 import io.netty.channel.ChannelInitializer; 4 import io.netty.channel.EventLoopGroup; 5 import io.netty.channel.nio.NioEventLoopGroup; 6 import io.netty.channel.socket.SocketChannel; 7 import io.netty.channel.socket.nio.NioServerSocketChannel; 8 import io.netty.handler.codec.http.HttpObjectAggregator; 9 import io.netty.handler.codec.http.HttpRequestDecoder; 10 import io.netty.handler.codec.http.HttpResponseEncoder; 11 import io.netty.handler.stream.ChunkedWriteHandler; 12 13 public class HttpFileServer { 14 15 public void run(int port, String folderPath) throws Exception { 16 EventLoopGroup bossGroup = new NioEventLoopGroup(); 17 EventLoopGroup workerGroup = new NioEventLoopGroup(); 18 try { 19 ServerBootstrap serverBootstrap = new ServerBootstrap(); 20 serverBootstrap.group(bossGroup, workerGroup) 21 .channel(NioServerSocketChannel.class) 22 .childHandler(new ChannelInitializer<SocketChannel>() { 23 24 @Override 25 protected void initChannel(SocketChannel socketChannel) throws Exception { 26 socketChannel.pipeline().addLast(new HttpRequestDecoder()); 27 socketChannel.pipeline().addLast(new HttpObjectAggregator(65536)); 28 socketChannel.pipeline().addLast(new HttpResponseEncoder()); 29 socketChannel.pipeline().addLast(new ChunkedWriteHandler()); 30 socketChannel.pipeline().addLast(new HttpFileServerHandler(folderPath)); 31 } 32 33 }); 34 ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); 35 channelFuture.channel().closeFuture().sync(); 36 } finally { 37 workerGroup.shutdownGracefully(); 38 bossGroup.shutdownGracefully(); 39 } 40 } 41 42 public static void main(String[] args) throws Exception { 43 int port = 8080; 44 String folderPath = "E:/workspace"; 45 new HttpFileServer().run(port, folderPath); 46 } 47 48 }
2. File server HttpFileServerHandler
1 import io.netty.buffer.ByteBuf; 2 import io.netty.buffer.Unpooled; 3 import io.netty.channel.ChannelFutureListener; 4 import io.netty.channel.ChannelHandlerContext; 5 import io.netty.channel.SimpleChannelInboundHandler; 6 import io.netty.handler.codec.http.DefaultFullHttpResponse; 7 import io.netty.handler.codec.http.DefaultHttpResponse; 8 import io.netty.handler.codec.http.FullHttpRequest; 9 import io.netty.handler.codec.http.FullHttpResponse; 10 import io.netty.handler.codec.http.HttpHeaders; 11 import io.netty.handler.codec.http.HttpMethod; 12 import io.netty.handler.codec.http.HttpResponse; 13 import io.netty.handler.codec.http.HttpResponseStatus; 14 import io.netty.handler.codec.http.HttpVersion; 15 import io.netty.handler.stream.ChunkedFile; 16 import io.netty.util.CharsetUtil; 17 18 import java.io.File; 19 import java.io.FileNotFoundException; 20 import java.io.RandomAccessFile; 21 import java.net.URLDecoder; 22 23 public class HttpFileServerHandler extends SimpleChannelInboundHandler<FullHttpRequest> { 24 25 private String folderPath; 26 27 public HttpFileServerHandler(String folderPath) { 28 this.folderPath = folderPath; 29 } 30 31 @Override 32 protected void messageReceived(ChannelHandlerContext ctx, FullHttpRequest req) throws Exception { 33 if (!req.getDecoderResult().isSuccess()) { 34 sendStatus(ctx, HttpResponseStatus.BAD_REQUEST); 35 return; 36 } 37 if (!HttpMethod.GET.equals(req.getMethod())) { 38 sendStatus(ctx, HttpResponseStatus.METHOD_NOT_ALLOWED); 39 return; 40 } 41 String uri = req.getUri(); 42 File file = getFile(uri); 43 if (file == null || file.isHidden() || !file.exists()) { 44 sendStatus(ctx, HttpResponseStatus.NOT_FOUND); 45 return; 46 } 47 try { 48 if (file.isDirectory()) { 49 listFiles(ctx, file, uri); 50 } else { 51 returnFile(ctx, req, file); 52 } 53 } catch (Exception e) { 54 sendStatus(ctx, HttpResponseStatus.INTERNAL_SERVER_ERROR); 55 } 56 } 57 58 private File getFile(String uri) throws Exception { 59 uri = URLDecoder.decode(uri, "UTF-8"); 60 return new File(folderPath + uri); 61 } 62 63 private void listFiles(ChannelHandlerContext ctx, File folder, String uri) throws Exception { 64 uri = uri.endsWith("/") ? uri : uri + "/"; 65 StringBuilder html = new StringBuilder("<h1>Index of ").append(URLDecoder.decode(uri, "UTF-8")).append("</h1><hr/><pre><a href=\"").append(uri).append("../\">../</a>\n"); 66 File[] subfiles = folder.listFiles(); 67 if (subfiles != null && subfiles.length > 0) { 68 for (File subfile : subfiles) { 69 String name = subfile.getName(); 70 html.append("<a href=\"").append(uri).append(name).append("\">").append(name).append("</a>\n"); 71 } 72 } 73 html.append("</pre><hr/>"); 74 FullHttpResponse resp = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK); 75 resp.headers().add(HttpHeaders.Names.CONTENT_TYPE, "text/html;charset=UTF-8"); 76 ByteBuf content = Unpooled.copiedBuffer(html, CharsetUtil.UTF_8); 77 resp.content().writeBytes(content); 78 ctx.writeAndFlush(resp).addListener(ChannelFutureListener.CLOSE); 79 } 80 81 private void returnFile(ChannelHandlerContext ctx, FullHttpRequest req, File file) throws Exception { 82 83 RandomAccessFile randomAccessFile = null; 84 try { 85 randomAccessFile = new RandomAccessFile(file, "r"); 86 HttpResponse resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK); 87 resp.headers().set(HttpHeaders.Names.CONTENT_LENGTH, randomAccessFile.length()) 88 .set(HttpHeaders.Names.CONTENT_TYPE, "application/octet-stream"); 89 if (HttpHeaders.Values.KEEP_ALIVE.toString().equalsIgnoreCase(req.headers().get(HttpHeaders.Names.CONNECTION))) { 90 resp.headers().set(HttpHeaders.Names.CONNECTION, HttpHeaders.Values.KEEP_ALIVE); 91 } 92 ctx.write(resp); 93 ctx.writeAndFlush(new ChunkedFile(randomAccessFile, 0, randomAccessFile.length(), 8192)).addListener(ChannelFutureListener.CLOSE); 94 95 } catch (FileNotFoundException e) { 96 sendStatus(ctx, HttpResponseStatus.NOT_FOUND); 97 } finally { 98 if (randomAccessFile != null) { 99 randomAccessFile.close(); 100 } 101 } 102 } 103 104 private void sendStatus(ChannelHandlerContext ctx, HttpResponseStatus status) throws Exception { 105 HttpResponse resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, status); 106 ctx.writeAndFlush(resp).addListener(ChannelFutureListener.CLOSE); 107 } 108 109 }
Development of WebSocket Protocol
Problems and Solutions
1. Round training, Comet and other server push technologies are inefficient and consume a lot of server bandwidth and resources.
2. The characteristics of WebSocket:
a) Single TCP connection, full duplex mode.
b) Transparent to agents, firewalls and routers.
c) No header information, cookies, and authentication.
d) No security overhead.
e) Maintain link activation by "ping/pong" frame.
f) The server can actively deliver messages to the client, and the client can no longer rotate training.
Principle (process)
1. The browser initiates an HTTP request to the server (special header information, Sec-WebSocket-Key is random), ready to establish a WebSocket connection.
GET /chat HTTP/1.1 Host: server.example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw== Sec-WebSocket-Protocol: chat, superchat Sec-WebSocket-Version: 13 Origin: http://example.com
2. Server uses Sec-WebSocket-Key and magic string "258EAFA5-E914-47DA-95CA-C5AB0DC85B11", first SHA-1 encryption, then BASE-64 encoding, as Sec-WebSocket-Accept back to the browser. Shake hands.
HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat
3. Server and browser can communicate by message.
4. The shutdown message has a status code and an optional shutdown reason. It sends a Close control frame according to the protocol requirements. When the opposite end receives the instruction of shutdown control frame, it actively closes the WebSocket connection.
Development
1. Server WebSocket Server
1 import io.netty.bootstrap.ServerBootstrap; 2 import io.netty.channel.ChannelFuture; 3 import io.netty.channel.ChannelInitializer; 4 import io.netty.channel.EventLoopGroup; 5 import io.netty.channel.nio.NioEventLoopGroup; 6 import io.netty.channel.socket.SocketChannel; 7 import io.netty.channel.socket.nio.NioServerSocketChannel; 8 import io.netty.handler.codec.http.HttpObjectAggregator; 9 import io.netty.handler.codec.http.HttpRequestDecoder; 10 import io.netty.handler.codec.http.HttpResponseEncoder; 11 import io.netty.handler.stream.ChunkedWriteHandler; 12 13 public class WebSocketServer { 14 15 public void run(int port) throws Exception { 16 EventLoopGroup bossGroup = new NioEventLoopGroup(); 17 EventLoopGroup workerGroup = new NioEventLoopGroup(); 18 try { 19 ServerBootstrap serverBootstrap = new ServerBootstrap(); 20 serverBootstrap.group(bossGroup, workerGroup) 21 .channel(NioServerSocketChannel.class) 22 .childHandler(new ChannelInitializer<SocketChannel>() { 23 24 @Override 25 protected void initChannel(SocketChannel socketChannel) throws Exception { 26 socketChannel.pipeline().addLast(new HttpRequestDecoder()); 27 socketChannel.pipeline().addLast(new HttpObjectAggregator(65536)); 28 socketChannel.pipeline().addLast(new HttpResponseEncoder()); 29 socketChannel.pipeline().addLast(new ChunkedWriteHandler()); 30 socketChannel.pipeline().addLast(new WebSocketServerHandler()); 31 } 32 33 }); 34 ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); 35 channelFuture.channel().closeFuture().sync(); 36 } finally { 37 workerGroup.shutdownGracefully(); 38 bossGroup.shutdownGracefully(); 39 } 40 } 41 42 public static void main(String[] args) throws Exception { 43 int port = 8080; 44 new WebSocketServer().run(port); 45 } 46 47 }
2. Server WebSocket Server Handler
1 import io.netty.channel.ChannelFutureListener; 2 import io.netty.channel.ChannelHandlerContext; 3 import io.netty.channel.SimpleChannelInboundHandler; 4 import io.netty.handler.codec.http.DefaultHttpResponse; 5 import io.netty.handler.codec.http.FullHttpRequest; 6 import io.netty.handler.codec.http.HttpHeaders; 7 import io.netty.handler.codec.http.HttpResponse; 8 import io.netty.handler.codec.http.HttpResponseStatus; 9 import io.netty.handler.codec.http.HttpVersion; 10 import io.netty.handler.codec.http.websocketx.CloseWebSocketFrame; 11 import io.netty.handler.codec.http.websocketx.PingWebSocketFrame; 12 import io.netty.handler.codec.http.websocketx.PongWebSocketFrame; 13 import io.netty.handler.codec.http.websocketx.TextWebSocketFrame; 14 import io.netty.handler.codec.http.websocketx.WebSocketFrame; 15 import io.netty.handler.codec.http.websocketx.WebSocketServerHandshaker; 16 import io.netty.handler.codec.http.websocketx.WebSocketServerHandshakerFactory; 17 18 import java.util.Date; 19 20 public class WebSocketServerHandler extends SimpleChannelInboundHandler<Object> { 21 22 private WebSocketServerHandshaker handshaker; 23 24 @Override 25 protected void messageReceived(ChannelHandlerContext ctx, Object msg) throws Exception { 26 // tradition HTTP 27 if (msg instanceof FullHttpRequest) { 28 handleHttpRequest(ctx, (FullHttpRequest) msg); 29 } else if (msg instanceof WebSocketFrame) { 30 handleWebSocketFrame(ctx, (WebSocketFrame) msg); 31 } 32 } 33 34 @Override 35 public void channelReadComplete(ChannelHandlerContext ctx) throws Exception { 36 ctx.flush(); 37 } 38 39 private void handleHttpRequest(ChannelHandlerContext ctx, FullHttpRequest req) throws Exception { 40 if (!req.getDecoderResult().isSuccess() 41 || !HttpHeaders.Values.WEBSOCKET.toString().equalsIgnoreCase(req.headers().get(HttpHeaders.Names.UPGRADE))) { 42 sendStatus(ctx, HttpResponseStatus.BAD_REQUEST); 43 return; 44 } 45 WebSocketServerHandshakerFactory wsFactory = new WebSocketServerHandshakerFactory("ws://localhost:8080/testws", null, false); 46 handshaker = wsFactory.newHandshaker(req); 47 if (handshaker == null) { 48 WebSocketServerHandshakerFactory.sendUnsupportedWebSocketVersionResponse(ctx.channel()); 49 } else { 50 handshaker.handshake(ctx.channel(), req); 51 } 52 } 53 54 private void handleWebSocketFrame(ChannelHandlerContext ctx, WebSocketFrame frame) throws Exception { 55 if (frame instanceof CloseWebSocketFrame) { 56 handshaker.close(ctx.channel(), (CloseWebSocketFrame) frame.retain()); 57 return; 58 } 59 if (frame instanceof PingWebSocketFrame) { 60 ctx.channel().write(new PongWebSocketFrame(frame.content().retain())); 61 return; 62 } 63 if (!(frame instanceof TextWebSocketFrame)) { 64 throw new UnsupportedOperationException(); 65 } 66 String req = ((TextWebSocketFrame) frame).text(); 67 ctx.channel().write(new TextWebSocketFrame("Welcome" + req + "，Now moment" + new Date())); 68 } 69 70 private void sendStatus(ChannelHandlerContext ctx, HttpResponseStatus status) throws Exception { 71 HttpResponse resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, status); 72 ctx.writeAndFlush(resp).addListener(ChannelFutureListener.CLOSE); 73 } 74 75 }
3. Browser websocketclient.html
1 <script type="text/javascript"> 2 var socket; 3 function initSocket() { 4 if (socket) return; 5 if (!window.WebSocket) window.WebSocket = window.MozWebSocket; 6 if (!window.WebSocket) { 7 alert('Browser does not support WebSocket'); 8 return; 9 } 10 socket = new WebSocket('ws://localhost:8080/testws'); 11 socket.onmessage = function(event) { 12 alert(event.data); 13 }; 14 socket.onopen = function(event) { 15 alert('WebSocket Successful Connection Establishment'); 16 }; 17 socket.onclose = function(event) { 18 alert('WebSocket Connection closed'); 19 }; 20 } 21 22 function sendMsg() { 23 initSocket(); 24 if (socket && WebSocket && socket.readyState == WebSocket.OPEN) { 25 var msg = document.getElementById('msg').value; 26 socket.send(msg); 27 } 28 } 29 </script> 30 <input type="text" id="msg"/> 31 <input type="button" value="Send" onclick="sendMsg()"/>
Netty architecture
Logical structure
1. Netty is designed and developed with a three-tier network architecture.
2. Reactor Communication Scheduling Layer (Layer 1). Responsible for monitoring network read-write and connection operations. Read the data from network layer into memory buffer, and then trigger various network events, such as connection creation, connection activation, read events, write events, etc. These events are triggered to Pipeline, and the responsibility chain managed by Pipeline is used for subsequent processing.
3. Chain of Responsibility Channel Pipleline (Layer 2). Responsible for the orderly dissemination of events in the chain of responsibility, while dynamically arranging the chain of responsibility. Usually, external protocol messages are converted into internal POJO objects by codec Handler, so the upper business only needs to care about business logic processing.
4. Service ChannelHandler, Business Logic Choreography Layer (Layer 3). There are usually two types: stored business logic choreography and other application layer protocol plug-ins for protocol-specific session and link management.
5. Usually, the developer value needs the responsibility chain and the business logic choreography layer.
High performance
How does Netty achieve high performance?
1. Using asynchronous non-blocking IO class library and based on Reactor mode, the problem that a server can not smoothly handle a linearly growing client in traditional synchronous blocking IO mode is solved.
2. TCP receiving and sending buffers use direct memory instead of heap memory to avoid memory replication and improve IO read-write performance. It is commonly known as Zero-Copy.
3. By using ByteBuf in memory pool, the performance loss caused by frequent creation and destruction of ByteBuf is avoided.
4. The number of IO threads and TCP parameters can be configured to provide customized tuning parameters for different scenarios to meet different performance scenarios.
5. Using ring array buffer to realize lock-free concurrent programming instead of traditional thread-safe containers and locks.
6. Reasonable use of thread-safe containers, atomic classes, etc. to enhance the concurrent processing capability of the system.
7. The processing of key resources uses single-thread serialization to avoid lock competition and additional CPU resource consumption caused by multi-thread concurrent access.
8. Fine-grained memory management reduces the frequency of GC and the latency and CPU loss caused by frequent GC by timely application of reference counters to release objects that are no longer referenced.
reliability
How does Netty's reliability work?
1. Link validity detection.
a) Long connections do not need to create links every time a message is sent, nor do they need to close links after the message interaction is completed, so they are higher than short links.
b) To ensure the validity of long connections, periodic heartbeat detection is needed. Once the problem is found, the link can be shut down and the TCP link can be reconstructed in time.
2. Memory protection mechanism.
(a) Fine-grained memory applications and releases for built-in objects such as ByteBuf through object reference counters to detect and protect illegal object references.
b) Reuse ByteBuf through memory pool to save memory.
c) The upper limit of memory capacity can be set, including ByteBuf, thread pool threads, etc.
3. Elegant shutdown.
a) When the system exits, JVM intercepts the exit semaphore through registered Hutdown Hook, then executes the exit operation, releases the resources of relevant modules, completes or empties the message processing in the buffer, persists the data to be refreshed to disk or database, and then exits after completion.
b) Time-out time T should be set. If the time-out time T has not been withdrawn after reaching T, kill-9 PID will be used to kill the process.
Customizable
How is Netty customizability achieved?
1. Responsibility chain model: Channel Pipeline is based on responsibility chain model, which facilitates the interception, customization and expansion of business logic.
2. Interface-based development: Key class libraries provide interfaces or abstract classes.
3. Provide a large number of factory classes, overload factory classes can create user-implemented objects.
4. Provide a large number of system parameters for users to set.
Extensibility
Private protocol stacks can be defined.
Private protocol stack development
1. Code written at the time of development.
(a) Data structure Netty Message;
b) Message codec NettyMessage Encoder and NettyMessage Decoder;
c) Handshake authentication Handler Login AuthReq Hanlder and Login AuthResp Hanlder;
d) Heart beat detection Handler HearBeat Req Hanlder and HearBeat Resp Hanlder.
2. Private protocol stack details need to be added.
Author: netoxi
Source: http://www.cnblogs.com/netoxi
Copyright of this article belongs to the author and the blog park. Reprint is welcome. This statement must be retained without permission, and the link of the original text is given in the obvious position on the article page. Welcome to correct and communicate.