Java Extensible IO

Doug Lee

outline

• Extensible Network Services
• event driven
• Reactor mode
  • Basic Edition
  • Multithreaded version
  • Other Variations
• Overview of blocking IO API s in java.io packages

Network Application Server

• Web server, distributed object system, etc.
• Their common features
  • Read Request
  • Decode Request Message
  • Business Processing
  • Coded response message
  • Send response
• Different steps in practical application
  • XML parsing
  • file transfer
  • Dynamic Web Page Generation
  • Computational Services

Classic Service Design

Run one handler per thread

Classic ServerSocket listening loop

class Server implements Runnable {
  public void run() {
    try {
      ServerSocket ss = new ServerSocket(PORT);
      while (!Thread.interrupted())
        new Thread(new Handler(ss.accept())).start();
        // Use a single thread or thread pool here
    } catch (IOException ex) { /* ... */ }
  }

  static class Handler implements Runnable {
    final Socket socket;
    Handler(Socket s) { socket = s; }
    public void run() {
        try {
            byte[] input = new byte[MAX_INPUT];
            socket.getInputStream().read(input);
            byte[] output = process(input);
            socket.getOutputStream().write(output);
          } catch (IOException ex) { /* ... */ }
    }
    private byte[] process(byte[] cmd) { /* ... */ }
  }
}

Note: Exception handling omission

High scalability goals

• Decrease in service elegance as pressure keeps increasing (more clients)
• Performance continues to increase as resources (CPU, memory, disk, bandwidth) increase
• Goals for high availability and performance
  • Low latency
  • Respond to peak requests
  • Quality of Service Controllable
• Divide and conquer are common solutions to scalability problems

Divide and rule

• Divide the process into smaller tasks, each task being non-blocking
• The IO event model is usually a trigger mechanism when the task is ready to execute
• The java.nio package contains basic non-blocking mechanisms
  • Non-blocking read and write
  • Distributing tasks to associate tasks with IO event models
• More ideas
  • Event Driven Design

    Event Driven Design

• Usually more efficient
  • Less resources do not require a thread to be started for each client
  • Reduce overhead, reduce context switching, and less lock competition
  • Event distribution is slow and action and time must be bound together
• Higher programming difficulty
  • The process must be split into a series of non-blocking task units
    • Similar to the GUI event model
    • Not all blockages can be removed, such as GC, memory page errors, etc.
  • Attention must be paid to changes in the state of the service

Event Driver for Java AWT Graphic Programming

Event-driven IO models are similar, but design is continuous

Reactor model

• Reacttor is responsible for distributing events to the corresponding handler, similar to an AWT thread
• Handlers are non-blocking tasks, similar to ActionListeners in AWT
• Manage is responsible for binding handler s to events
• See Pattern-Oriented Software Architecture Volume 2 by Schmidt et al. And Richard Stevens about the number of networks, MattWelsh's SEDA framework, and so on

Reactor Basic

Single Thread Edition

Support provided by java.nio

• Channels

  >Channels are connections between files and socket s, supporting non-blocking reads

• Buffers

  >Array object, which can be read and written directly by Channel

• Selectors >Responsible for filtering those Channel IO events
• SelectionKeys >Save IO event state and bound objects

Reactor 1: Create

class Reactor implements Runnable {
    final Selector selector;
    final ServerSocketChannel serverSocket;
    Reactor(int port) throws IOException {
        selector = Selector.open();
        serverSocket = ServerSocketChannel.open(); 
        serverSocket.socket().bind( new InetSocketAddress(port) ); 
        serverSocket.configureBlocking(false);
        SelectionKey sk = serverSocket.register( selector, SelectionKey.OP_ACCEPT );
        sk.attach(new Acceptor());
    }
    /*
        You can also use SPI provider s, more specifically: 
        SelectorProvider p = SelectorProvider.provider(); 
        selector = p.openSelector();
        serverSocket = p.openServerSocketChannel();
    */

Reactor 2: Distribution Cycle

    public void run() {  // Usually start a new thread
        try {
            while (!Thread.interrupted()) {
                selector.select();
                Set selected = selector.selectedKeys(); 
                Iterator it = selected.iterator(); 
                while (it.hasNext())
                    dispatch((SelectionKey)(it.next()); 
                selected.clear();
            }
        } catch (IOException ex) { /* ... */ }
    }

    void dispatch(SelectionKey k) {
        Runnable r = (Runnable)(k.attachment()); 
        if (r != null) 
            r.run();
    }

Reactor 3: Acceptor

    class Acceptor implements Runnable { // Built-in class
        public void run() {
            try {
                SocketChannel c = serverSocket.accept();
                if (c != null)
                    new Handler(selector, c);
            } catch (IOException ex) { /* ... */ }
        } 
    }
}

Reactor 4: Build Handler

final class Handler implements Runnable {
    final SocketChannel socket;
    final SelectionKey sk;
    ByteBuffer input = ByteBuffer.allocate(MAXIN); 
    ByteBuffer output = ByteBuffer.allocate(MAXOUT); 
    static final int READING = 0, SENDING = 1;
    int state = READING;
    Handler(Selector sel, SocketChannel c) throws IOException {
        socket = c; 
        c.configureBlocking(false);
        // Attempt to listen for events
        sk = socket.register(sel, 0); 
        sk.attach(this); 
        sk.interestOps(SelectionKey.OP_READ); 
        sel.wakeup();
    }
    boolean inputIsComplete() { /* ... */ } 
    boolean outputIsComplete() { /* ... */ } 
    void process() { /* ... */ }

Reactor 5: Processing requests

    public void run() {
        try {
            if (state == READING) read();
            else if (state == SENDING) send();
        } catch (IOException ex) { /* ... */ }
    }

    void read() throws IOException {
        socket.read(input);
        if (inputIsComplete()) {
            process();
            state = SENDING;
            // After reading, usually listen for events
            sk.interestOps(SelectionKey.OP_WRITE);
        } 
    }

    void send() throws IOException { 
        socket.write(output);
        if (outputIsComplete()) 
            sk.cancel();
    } 
}

Handlers State Flow

Simple application of GoF state mode

class Handler { // ...
    public void run() { // The initial state is read
        socket.read(input);
        if (inputIsComplete()) {
            process();
            sk.attach(new Sender()); 
            sk.interest(SelectionKey.OP_WRITE); 
            sk.selector().wakeup();
        }
    }

    class Sender implements Runnable { 
        public void run(){ // ...
            socket.write(output);
            if (outputIsComplete()) 
                sk.cancel(); 
        }
    }
}

Multithreaded version

• Start the appropriate number of threads, especially in multi-core scenarios

• Work Threads

  • Reactors must trigger handlers quickly

    > The Handler task is heavy and will slow down the Reactor

  • Delegate computational work to another thread

• Multiple Readctor Threads

  • Reactor threads focus on IO operations

  • Allocate pressure to other reactor s

    >Load balancing considers adapting CPU and IO rates

Worker Threads

• Processing only computational tasks to speed up Reactor threads

  >Similar to Proactor mode in POSA2

• Easier than changing to event-driven mode

  >Processing only computational tasks

• Duplicate IO is more difficult to handle

  >Better to read it all out to the cache the first time you read it

• Use thread pool for easy control

  >Usually only a small number of threads, much less than the number of clients

Worker Thread Pool

Andler with Thread Pool

class Handler implements Runnable {
    // Using thread pools in util.concurrent
    static PooledExecutor pool = new PooledExecutor(...);
    static final int PROCESSING = 3;
    // ...
    synchronized void read() { // ...
        socket.read(input);
        if (inputIsComplete()) {
            state = PROCESSING;
            pool.execute(new Processer());
        }
    }
    synchronized void processAndHandOff() {
        process();
        state = SENDING; // or rebind attachment
        sk.interest(SelectionKey.OP_WRITE);
    }

    class Processer implements Runnable {
        public void run() { processAndHandOff(); }
    }
}

How to invoke a task

• Chain transfer

  >Each task is responsible for invoking the next task, which is usually the most efficient but prone to problems

• Distributor callback to each handler

  >By setting state, binding objects, etc., variants of GoF Mediator mode

• queue

  >As in the example above, data in buffer s is passed through a queue

• Future

  >The caller gets the execution results of each task through the join or wait/notify methods

Using PooledExecutor

• Schedule worker thread
• Main method: execute(Runnable r)

• There are the following control parameters

  • Queue type
  • Maximum Threads
  • Minimum Threads
  • "Warm" versus on-demand threads

  • Automatically reclaim idle threads

    >Create a new thread when needed

  • Multiple strategies for task saturation

    >Blocking, discarding, producer-runs, etc.

Multiple Reactor Threads

• Use Reactor pool

  • Adapt CPU and IO rates

  • Static or dynamic creation

    >Each reactor has its own Selector, Thread, Distribution Loop

  • Master acceptor distributed to other reactors

>

Selector[] selectors;
int next = 0;
class Acceptor { // ...
    public synchronized void run() { ...
        Socket connection = serverSocket.accept(); 
        if (connection != null)
            new Handler(selectors[next], connection); 
        if (++next == selectors.length) 
            next = 0;
    } 
}

Multiple Reactor Example

Other java.nio features

• Each Reactor contains multiple Selector s

  >Be careful about synchronization issues when binding different handler s to different IO events

• file transfer

  >Automatic file transfer: file-to-net or net-to-file replication

• Memory Mapping File

  >Access files through buffers

• Direct access buffer

  >Sometimes zero copy can be achieved

  > But have setup and finalization overhead

  >Ideal for long-connection applications

Extend Network Connections

• Receive multiple requests at the same time

  >Client Connections

  >Client sends a series of messages/requests

  >Client Disconnect

• For instance

  >Database Transaction Monitor

  >Multiplayer online games, chat rooms, etc.

• Extend the underlying network model above

  >Keep many relatively long-lived clients alive

  >Track clients, keep session state (including discards)

  >Distributed services, across multiple hosts

API Overview

• Buffer

• ByteBuffer

  > (CharBuffer, LongBuffer, etc.)

• Channel
• SelectableChannel
• SocketChannel
• ServerSocketChannel
• FileChannel
• Selector
• SelectionKey

Buffer

abstract class Buffer {
  int     capacity();
  int     position();
  Buffer  position(int newPosition);
  int     limit();
  Buffer  limit(int newLimit);
  Buffer  mark();
  Buffer  reset();
  Buffer  clear();
  Buffer  flip();
  Buffer  rewind();
  int     remaining();
  boolean hasRemaining();
  boolean isReadOnly();
}

ByteBuffer

abstract class ByteBuffer extends Buffer {
    static ByteBuffer allocateDirect(int capacity);
    static ByteBuffer allocate(int capacity);
    static ByteBuffer wrap(byte[] src, int offset, int len);
    static ByteBuffer wrap(byte[] src);
    boolean      isDirect();
    ByteOrder    order();
    ByteBuffer   order(ByteOrder bo);
    ByteBuffer   slice();
    ByteBuffer   duplicate();
    ByteBuffer   compact();
    ByteBuffer   asReadOnlyBuffer();

    byte        get();
    byte        get(int index);
    ByteBuffer  get(byte[] dst, int offset, int length);
    ByteBuffer  get(byte[] dst);
    ByteBuffer  put(byte b);
    ByteBuffer  put(int index, byte b);
    ByteBuffer  put(byte[] src, int offset, int length);
    ByteBuffer  put(ByteBuffer src);
    ByteBuffer  put(byte[] src);
    char        getChar();
    char        getChar(int index);
    ByteBuffer  putChar(char value);
    ByteBuffer  putChar(int index, char value); 
    CharBuffer  asCharBuffer();
    short           getShort();
    short           getShort(int index);
    ByteBuffer      putShort(short value);
    ByteBuffer      putShort(int index, short value);
    ShortBuffer  asShortBuffer();
    int          getInt();
    int          getInt(int index);
    ByteBuffer   putInt(int value);
    ByteBuffer   putInt(int index, int value);
    IntBuffer    asIntBuffer();
    long         getLong();
    long         getLong(int index);
    ByteBuffer   putLong(long value);
    ByteBuffer   putLong(int index, long value);
    LongBuffer   asLongBuffer();
    float       getFloat();
    float       getFloat(int index);
    ByteBuffer  putFloat(float value);
    ByteBuffer  putFloat(int index, float value);
    FloatBuffer  asFloatBuffer();
    double      getDouble();
    double      getDouble(int index);
    ByteBuffer  putDouble(double value);
    ByteBuffer  putDouble(int index, double value);
    DoubleBuffer asDoubleBuffer();
}

Channel

interface Channel {
  boolean isOpen();
  void    close() throws IOException;
}
interface ReadableByteChannel extends Channel {
  int     read(ByteBuffer dst) throws IOException;
}
interface WritableByteChannel  extends Channel {
  int     write(ByteBuffer src) throws IOException;
}
interface ScatteringByteChannel extends ReadableByteChannel {
  int     read(ByteBuffer[] dsts, int offset, int length) throws IOException;
  int     read(ByteBuffer[] dsts) throws IOException;
}
interface GatheringByteChannel extends WritableByteChannel {
  int     write(ByteBuffer[] srcs, int offset, int length) throws IOException;
  int     write(ByteBuffer[] srcs) throws IOException;
}

SelectableChannel

abstract class SelectableChannel implements Channel {
    int          validOps();
    boolean      isRegistered();
    SelectionKey keyFor(Selector sel);
    SelectionKey register(Selector sel, int ops) throws ClosedChannelException;
    void        configureBlocking(boolean block) throws IOException;
    boolean     isBlocking();
    Object      blockingLock();
}

SocketChannel

abstract class SocketChannel implements ByteChannel ... {
  static SocketChannel open() throws IOException;
  Socket  socket();
  int     validOps();
  boolean isConnected();
  boolean isConnectionPending();
  boolean isInputOpen();
  boolean isOutputOpen();
  boolean connect(SocketAddress remote) throws IOException;
  boolean finishConnect() throws IOException;
  void    shutdownInput() throws IOException;
  void    shutdownOutput() throws IOException;
  int     read(ByteBuffer dst) throws IOException;
  int     read(ByteBuffer[] dsts, int offset, int length)
              throws IOException;
  int     read(ByteBuffer[] dsts) throws IOException;
  int     write(ByteBuffer src) throws IOException;
  int     write(ByteBuffer[] srcs, int offset, int length)
              throws IOException;
  int     write(ByteBuffer[] srcs) throws IOException;
}

ServerSocketChannel

abstract class ServerSocketChannel extends ... {
  static ServerSocketChannel open() throws IOException;
  int           validOps();
  ServerSocket  socket();
  SocketChannel accept() throws IOException;
}

FileChannel

abstract class FileChannel implements ... {
  int  read(ByteBuffer dst);
  int  read(ByteBuffer dst, long position);
  int  read(ByteBuffer[] dsts, int offset, int length);
  int  read(ByteBuffer[] dsts);
  int  write(ByteBuffer src);
  int  write(ByteBuffer src, long position);
  int  write(ByteBuffer[] srcs, int offset, int length);
  int  write(ByteBuffer[] srcs);
  long position();
  void position(long newPosition);
  long size();
  void truncate(long size);
  void force(boolean flushMetaDataToo);
  int  transferTo(long position, int count,
                  WritableByteChannel dst);
  int  transferFrom(ReadableByteChannel src,
                    long position, int count);
  FileLock lock(long position, long size, boolean shared);
  FileLock lock();
  FileLock tryLock(long pos, long size, boolean shared);
  FileLock tryLock();
  static final int MAP_RO, MAP_RW,  MAP_COW;
  MappedByteBuffer map(int mode, long position, int size);
}
NOTE: All methods are thrown IOException

Selector

abstract class Selector {
  static Selector open() throws IOException;
  Set  keys();
  Set  selectedKeys();
  int  selectNow() throws IOException;
  int  select(long timeout) throws IOException;
  int  select() throws IOException;
  void wakeup();
  void close() throws IOException;
}

SelectionKey

abstract class SelectionKey {
    static final int  OP_READ, OP_WRITE, OP_CONNECT, OP_ACCEPT;
    SelectableChannel channel();
    Selector    selector();
    boolean     isValid();
    void    cancel();
    int     interestOps();
    void    interestOps(int ops);
    int     readyOps();
    boolean isReadable();
    boolean isWritable();
    boolean isConnectable();
    boolean isAcceptable();
    Object  attach(Object ob);
    Object  attachment();
}