Nio learning

Keywords: Java JDK socket github

Nio learning

The article is a summary of my own study. If there is any misunderstanding, please leave a message.

What is Nio?

The full name of java.nio is java non-blocking IO (actually new io), which refers to the new api (New IO) provided in jdk1.4 and above for all primitive types (except boolean types). cache Supported data container, which can provide non-blocking high scalability network.

 

What is blocking and what is non-blocking?

  • Traditional IO: Traditional IO is blocking. When a server wants to read data from a file system, it needs to set up a thread to read. But at the beginning of reading, it may be that the file system is not ready for the data, but the thread can only wait for the file system to prepare the data before reading. It can't do anything else first. This is the blockage.
  • NIO: Nio can solve the blocking problem in traditional IO. It uses Selector (selector, which will be explained later) to monitor whether the data is ready. When the data is ready, the server allocates threads for the read operation. In this way, the server can make good use of the only thread resources.

It's like booking a car on a drop in your life, estimating that the 8:00 driver will arrive, and you'll be waiting at 7:50 at the gate of the community, so you'll have 10 minutes waiting time, which is congestion. Instead, you wait for the 8:00 driver to arrive and call to inform you that you have arrived at the gate of the community, then you go out again. If you save 10 minutes, you can choose to do something meaningful, which is non-blocking.

But please note that there will be no blockage without NIO!!! It's not that NIO won't block up!!! It's not that NIO won't block up!!! Important things are to be repeated for 3 times. In NIO, there are blocking and non-blocking, which will be discussed later.

 

Three components in NIO that you should know about

There are three important concepts in NIO, (o)... If you don't remember, I'm afraid you can't keep looking down.

  • Buffer: Buffer, a container for storing data. It's actually an array.
  • channel: channel that represents the connection between IO sources and Applications
  • Selector: Selector. If channel is registered in the selector, the selector can listen on the channel. A selector can monitor multiple channels.

Buffer can be understood as a train, Channel can be understood as a railway, and buffer runs in channel. So I can conclude that channel does not store data!!!

Channel reads and writes data from buffer s every time

buffer learning( java.nio.Buffer)

buffer is actually an array. Buffers can be of many types. Basic types in java can be associated with buffers (except boolean).

Suddenly, I felt that boolean was so pitiful that people didn't take it to play with them. The most used one was probably Byte Buffer.

There are three important concepts in Buffer: capacity, limitation and location.

  • position: Marks the location of the current operand
  • Limit: Represents the size of the data that can be manipulated in the buffer, and the data after limit cannot be read or written.
  • Capacity: Marks the current capacity size

 

For example, I initialize a

ByteBuffer buffer = new ByteBuffer.allocate(5);

(allocate specifies the size of the buffer buffer. ) So the relationship between position,limit,capacith is as follows

 

It's too difficult to draw a picture. My concubines can't do it!!! At this time I

String str = "123";
buffer.put(str.getBytes());

Then position will move to the third grid, limit,capacity will remain unchanged.

But if you switch buffer to read mode

buffer.flip()

So the current position: 3, limit: 3, capacity: 5

    @Test
    public void test() {
        String str = "123";
        //Appoint buffer Size of capacity
        ByteBuffer buffer = ByteBuffer.allocate(1024);
        //attribute
        System.out.println("-----------------------");
        //Location of data in current operation
        System.out.println(buffer.position());
        //Limits, which indicate the size of data that can be manipulated in a buffer. limit Later data cannot be read or written
        System.out.println(buffer.limit());
        //Maximum storage capacity in buffer
        System.out.println(buffer.capacity());
        
        buffer.put(str.getBytes());
        System.out.println("---------put--------------");
        System.out.println(buffer.position());
        System.out.println(buffer.limit());
        System.out.println(buffer.capacity());
        //Switch to Read Mode
        buffer.flip();
        System.out.println("---------flip--------------");
        System.out.println(buffer.position());
        System.out.println(buffer.limit());
        System.out.println(buffer.capacity());
        
        byteArrs = new byte[buffer.limit()];
        ByteBuffer byteBuffer = buffer.get(byteArrs);
        System.out.println(byteBuffer.toString());
        // rewind,Switch to Read Mode and Read Again
        buffer.rewind();
        System.out.println("===========rewind============");
        System.out.println(buffer.toString());
        
        //Clear the buffer
        buffer.clear();
    }

Channel(java.nio.channels.Channel)

 

The main implementation classes in channel: FileChannel, Socket Channel, Server Socket Channel.

Method for obtaining Channel:

Prior to JDK 1.7: Access to channel via IO stream

    /**
     * Using Channel to Complete File Replication
     * @throws IOException 
     */
    @Test
    public void test() {
        long start = System.currentTimeMillis();
        FileInputStream fis = null;
        FileOutputStream fos = null;
        FileChannel inChannel = null;
        FileChannel outChannel = null;
        try {
            //jdk1.7 before NIO Writing of access channel
            fis  = new FileInputStream("./resource/Java NIO.pdf");
            fos = new FileOutputStream("./resource/demoCopyTest.jpeg");
            
            //1.Access channel
            inChannel = fis.getChannel();
            outChannel = fos.getChannel();
            //2.Creating Buffers,And assign size
            ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
            //3.Write data into the buffer
            while (inChannel.read(byteBuffer) != -1) {
                
                //4.Switching Read Data Mode
                byteBuffer.flip();
                outChannel.write(byteBuffer);
                byteBuffer.clear();
            }
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } finally {
            try {
                if (outChannel != null) {
                    outChannel.close();
                }
                if (inChannel != null) {
                    inChannel.close();
                }
                if (fos != null) {
                    fos.close();
                }
                if (fis != null) {
                    fis.close();
                }
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
        long end = System.currentTimeMillis();
        System.out.println("Time-consuming indirect caching" + (end - start));
    }
JDK1.7 before

After JDK 1.7: channel can be obtained directly through Open method

    @Test
    public void test2() {
        long start = System.currentTimeMillis();
        FileChannel inChannel = null;
        FileChannel outChannel = null;
        
        try {
            //Establishing channels
            inChannel = FileChannel.open(Paths.get("./resource/Java NIO.pdf"), StandardOpenOption.READ);
            outChannel = FileChannel.open(Paths.get("./resource/Java NIOCopyTest2.pdf"), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE);
            
        //    inChannel.transferTo(0, inChannel.size(), outChannel);
            outChannel.transferFrom(inChannel, 0, inChannel.size());
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } finally {
            try {
                if (outChannel != null) {
                    outChannel.close();
                }
                if (inChannel != null) {
                    inChannel.close();
                }
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

        }
        long end = System.currentTimeMillis();
        System.out.println("Time-consuming direct buffer" + (end - start));
    }
FileChannel.open
    /**
     * Direct buffer, done with memory mapped files
     * Problem you may encounter: The file has been copied, but the program may not be completed. We can only control when to write the mapping file, but we can't control when to write from the mapping file to disk.
     */
    @Test
    public void test1() {
        long start = System.currentTimeMillis();
        FileChannel inChannel = null;
        FileChannel outChannel = null;
        MappedByteBuffer inMap = null;
        MappedByteBuffer outMap = null;
        
        try {
            //Establishing channels
            inChannel = FileChannel.open(Paths.get("./resource/Java NIO.pdf"), StandardOpenOption.READ);
            outChannel = FileChannel.open(Paths.get("./resource/Java NIOCopyTest2.pdf"), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE);
            //Because it's a memory file mapping, we don't need to read streams.,Memory Mapping File
            //MappedByteBuffer Amount to allocateDriect()
            inMap = inChannel.map(MapMode.READ_ONLY, 0, inChannel.size());
            outMap = outChannel.map(MapMode.READ_WRITE, 0, inChannel.size());
            byte[] bytes = new byte[inMap.limit()];
            inMap.get(bytes);
            outMap.put(bytes);
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } finally {
            try {
                if (outChannel != null) {
                    outChannel.close();
                }
                if (inChannel != null) {
                    inChannel.close();
                }
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

        }
        long end = System.currentTimeMillis();
        System.out.println("Time-consuming direct buffer" + (end - start));
    }

selector(java.nio.channels.Selector )

Using Selector to achieve non-blocking, create selector

Selector selector = Selector.open();

 

Blocking NIO and non-blocking NIO

How does NIO implement non-blocking IO?

Uh huh. We still have to look at a picture of this problem. The blocking approach is that the client establishes a connection directly with the server without the need for an intermediate listener.

No Selector, so it's still blocking NIO

    @Test
    public void serverTest() {
        ServerSocketChannel serverSocketChannel = null;
        FileChannel fileChannel = null;
        SocketChannel socketChannel = null;
        try {
            serverSocketChannel = ServerSocketChannel.open();
            serverSocketChannel.bind(new InetSocketAddress(9898));
            socketChannel = serverSocketChannel.accept();
            ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
            fileChannel = FileChannel.open(Paths.get("./resource/blockTest.jpeg"), StandardOpenOption.WRITE, StandardOpenOption.CREATE);

            while (socketChannel.read(byteBuffer) != -1) {
                byteBuffer.flip();
                System.out.println(byteBuffer);
                fileChannel.write(byteBuffer);
                byteBuffer.clear();
            }
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } finally {
            try {
                if (fileChannel != null) {
                    fileChannel.close();
                }
                if (serverSocketChannel != null) {
                    serverSocketChannel.close();
                }
                if (socketChannel != null) {
                    socketChannel.close();
                }
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
    }

Non-blocking is like this.

 

 

When a client sends a request, it first arrives at the Selector, which is blocked like a wall between the client and server segments. Register channel into selector while sending the request. Selector listens for channel status

Channels are divided into four states:

public static final int OP_READ = 1 << 0; 
public static final int OP_WRITE = 1 << 2; 
public static final int OP_CONNECT = 1 << 3; 
public static final int OP_ACCEPT = 1 << 4;

Take Socket Channel for example, server side

    @Test
    public void server() {
        ServerSocketChannel serverSocketChannel = null;
        try {
            serverSocketChannel = ServerSocketChannel.open();
            serverSocketChannel.configureBlocking(false);
            //Establish Selector object
            Selector selector = Selector.open();
            //hold serverSocketChannel Hand Selector Manage and bind listening status OP_ACCEPT
            serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
            serverSocketChannel.bind(new InetSocketAddress(9898));
            while (selector.select() > 0) {
                //Obtaining an iterator
                Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
                while (iterator.hasNext()) {
                    SelectionKey selectionKey = iterator.next();
                    //judge channel Is it? is ready to accept Get ready
                    if (selectionKey.isAcceptable()) {
                        //Server connection request!!! It's time to connect, not like blocking. Io In that case, no matter 3720, the connection request is always connected.
                        SocketChannel socketChannel = serverSocketChannel.accept();
                        //Set to non-blocking
                        socketChannel.configureBlocking(false);
                        //register channel reach selector,Notice here that SocketChannel It's a new channel and needs to be registered.
                        //Monitor read
                        socketChannel.register(selector, SelectionKey.OP_READ);
                    } else if (selectionKey.isReadable()){
                        //Gets Ready in the Current Selector channel
                        SocketChannel socketChannel = (SocketChannel)selectionKey.channel();
                        ByteBuffer buffer = ByteBuffer.allocate(1024);
                        int len = 0;
                        while ((len = socketChannel.read(buffer) )!= -1) {
                            buffer.flip();
                            //Here you can pass the client over. byte Make some transformations
                            System.out.println(buffer);
                            buffer.clear();
                        }
                    }
                    //Remove completed events from iterators
                    iterator.remove();
                }
            }
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

Demo github address 

Posted by Chevy on Thu, 31 Jan 2019 20:51:16 -0800