JDK_Packages_java_nio

jdk jdk packages nio

发布时间 : 2019-09-26 10:39

字数:2.6k 阅读 :

年少风雅鲜衣怒马也不过一刹那 —《红昭愿》

从JDK4 开始 ,NIO是New I/O的简称，具有以下特性：

为所有的原始类型提供（Buffer）缓存支持；(IO也有BufferInputStream)
增加通道（channel）对象，作为新的原始 I/O 抽象；
提供了基于 Selector 的异步网络 I/O。(线程模型)
使用 java.nio.charset.Charset 作为字符集编码解码解决方案；
支持锁和内存映射(MappedByteBuffer)文件的文件访问接口；

参考资料:

http://tutorials.jenkov.com/java-nio/index.html

https://ifeve.com/overview/

https://docs.oracle.com/javase/8/docs/api/java/nio/package-summary.html

https://cloud.tencent.com/developer/information/%E4%BD%BF%E7%94%A8ByteBuffer

Overivew

Java NIO (New IO) is an alternative IO API for Java (from Java 1.4), meaning alternative to the standard Java IO and Java Networking API’s. Java NIO offers a different way of working with IO than the standard IO API’s.

Java NIO consist of the following core components:

Channels
Buffers
Selectors

Buffer

buffer api介绍

filed

capacity：容量，表示buffer的最大数据容量，缓冲区容量不能为负，创建后不能更改
limit：限制，按照索引来，limit之后的数据不可读写，即只有在limit范围内的数据我们才可以读写操作。
position：位置，下一个要读取或写入的索引位置。该位置不能大于limit的限制。
mark：标记，标记后的索引位置，我们可以通过reset方法恢复position到该位置，该mark的位置要小于或等于position

// Invariants: mark <= position <= limit <= capacity
private int mark = -1;
private int position = 0;
private int limit;
private int capacity;

method

clear : Clears this buffer. The position is set to zero, the limit is set to the capacity, and the mark is discarded.

将position置为0，并不清除buffer内容。
flip(翻转 write->read) : Flips this buffer. The limit is set to the current position and then the position is set to zero. If the mark is defined then it is discarded.

写完数据(put)，需要开始读的时候，将postion复位到0，并将limit设为当前postion。

Rewind(倒带,和flip相比少了对limit的操作) : Rewinds this buffer. The position is set to zero and the mark is discarded.
Remaining\hasRemaining : Returns the number of elements between the current position and the limit.

source

                        /**
     * Clears this buffer.  The position is set to zero, the limit is set to
     * the capacity, and the mark is discarded.
     *
     * <p> Invoke this method before using a sequence of channel-read or
     * <i>put</i> operations to fill this buffer.  For example:
     *
     * <blockquote><pre>
     * buf.clear();     // Prepare buffer for reading
     * in.read(buf);    // Read data</pre></blockquote>
     *
     * <p> This method does not actually erase the data in the buffer, but it
     * is named as if it did because it will most often be used in situations
     * in which that might as well be the case. </p>
     *
     * @return  This buffer
     */
    public final Buffer clear() {
        position = 0;
        limit = capacity;
        mark = -1;
        return this;
    }

    /**
     * Flips this buffer.  The limit is set to the current position and then
     * the position is set to zero.  If the mark is defined then it is
     * discarded.
     *
     * <p> After a sequence of channel-read or <i>put</i> operations, invoke
     * this method to prepare for a sequence of channel-write or relative
     * <i>get</i> operations.  For example:
     *
     * <blockquote><pre>
     * buf.put(magic);    // Prepend header
     * in.read(buf);      // Read data into rest of buffer
     * buf.flip();        // Flip buffer
     * out.write(buf);    // Write header + data to channel</pre></blockquote>
     *
     * <p> This method is often used in conjunction with the {@link
     * java.nio.ByteBuffer#compact compact} method when transferring data from
     * one place to another.  </p>
     *
     * @return  This buffer
     */
    public final Buffer flip() {
        limit = position;
        position = 0;
        mark = -1;
        return this;
    }
    /**
     * Rewinds this buffer.  The position is set to zero and the mark is
     * discarded.
     *
     * <p> Invoke this method before a sequence of channel-write or <i>get</i>
     * operations, assuming that the limit has already been set
     * appropriately.  For example:
     *
     * <blockquote><pre>
     * out.write(buf);    // Write remaining data
     * buf.rewind();      // Rewind buffer
     * buf.get(array);    // Copy data into array</pre></blockquote>
     *
     * @return  This buffer
     */
    public final Buffer rewind() {
        position = 0;
        mark = -1;
        return this;
    }
    /**
     * Returns the number of elements between the current position and the
     * limit.
     *
     * @return  The number of elements remaining in this buffer
     */
    public final int remaining() {
        return limit - position;
    }

    /**
     * Tells whether there are any elements between the current position and
     * the limit.
     *
     * @return  <tt>true</tt> if, and only if, there is at least one element
     *          remaining in this buffer
     */
    public final boolean hasRemaining() {
        return position < limit;
    }

buffer 继承体系

Here is a list of the core Buffer implementations in Java NIO:

ByteBuffer
CharBuffer
DoubleBuffer
FloatBuffer
IntBuffer
LongBuffer
ShortBuffer

These Buffer‘s cover the basic data types that you can send via IO: byte, short, int, long, float, double and characters.

ByteBuffer

ByteBuffer最核心的方法是put(byte)和get()。分别是往ByteBuffer里写一个字节，和读一个字节。

值得注意的是，ByteBuffer的读写模式是分开的，正常的应用场景是：往ByteBuffer里写一些数据，然后flip()，然后再读出来。

allocate

                    public static ByteBuffer allocateDirect(int capacity) {
        return new DirectByteBuffer(capacity);
    }
    public static ByteBuffer allocate(int capacity) {
        if (capacity < 0)
            throw new IllegalArgumentException();
        return new HeapByteBuffer(capacity, capacity);
    }

HeapByteBuffer

间接的ByteBuffer是在JVM的堆上面的。间接缓冲区就是我们通常说的堆缓冲区。

class HeapByteBuffer extends ByteBuffer {}

DirectByteBuffer

直接就是指MappedByteBuffer，直接使用内存映射（java的话就意味着在JVM之外分配虚拟地址空间）；

class DirectByteBuffer extends MappedByteBuffer implements DirectBuffer {}

资料:

https://cloud.tencent.com/developer/article/1428920 MappedByteBuffer VS FileChannel ，孰强孰弱

https://cloud.tencent.com/developer/article/1152616 堆外内存之 DirectByteBuffer 详解

MappedByteBuffer

load : 会整个文件加载到内存中。
isLoaded : 判断一个被映射的文件是否完全加载内存了
force :该方法会强制将此缓冲区上的任何更改写入映射到永久磁盘存储器上。

public abstract class MappedByteBuffer extends ByteBuffer {
    public final MappedByteBuffer load( )
    public final boolean isLoaded( )
    public final MappedByteBuffer force( )
}

Channels

Typically, all IO in NIO starts with a Channel. A Channel is a bit like a stream. From the Channel data can be read into a Buffer. Data can also be written from a Buffer into a Channel. Here is an illustration of that:

Java NIO: Channels and Buffers

Java NIO: Channels read data into Buffers, and Buffers write data into Channels

Java NIO Channels are similar to streams with a few differences:

You can both read and write to a Channels. Streams are typically one-way (read or write).
Channels can be read and written asynchronously.
Channels always read to, or write from, a Buffer.

Channel Implementations

Here are the most important Channel implementations in Java NIO:

FileChannel : reads data from and to files.

public abstract long transferTo(long position,long count,WritableByteChannel target)
throws IOException
This method is potentially much more efficient than a simple loop that reads from this channel and writes to the target channel. Many operating systems can transfer bytes directly from the filesystem cache to the target channel without actually copying them.
DatagramChannel : can read and write data over the network via UDP.
SocketChannel : can read and write data over the network via TCP.
ServerSocketChannel : allows you to listen for incoming TCP connections, like a web server does. For each incoming connection a SocketChannel is created.

Selectors

Why Use a Selector?

A Selector allows a single thread to handle multiple Channel‘s. This is handy if your application has many connections (Channels) open, but only has low traffic on each connection. For instance, in a chat server.

Here is an illustration of a thread using a Selector to handle 3 Channel‘s:

Java NIO: Selectors

Java NIO: A Thread uses a Selector to handle 3 Channel’s

To use a Selector you register the Channel‘s with it. Then you call it’s select() method. This method will block until there is an event ready for one of the registered channels. Once the method returns, the thread can then process these events. Examples of events are incoming connection, data received etc.

full demo

Server demo

package nio;

import org.springframework.util.StringUtils;

import java.io.Closeable;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;

/**
 * @author zhengyumin
 * @description Nio服务端 使用selector接收消息
 * @date 2019-09-25 1:22 AM
 */
public class NioServer implements Closeable{

    private Selector selector;

    private ServerSocketChannel serverSocketChannel;

    private int port;

    private String hostname;

    public NioServer(int port) {
        this.port = port;
    }

    public NioServer(int port, String hostname) {
        this.port = port;
        this.hostname = hostname;
    }

    public static NioServer port(int port) {
        return new NioServer(port);
    }

    public NioServer start() {

        System.out.println("starting....");

        try {
            startServer();
            //开启selector 并监听
            startSelector();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return this;
    }

    @Override
    public void close() {
        try {
            System.out.println("closing....");
            selector.close();
            serverSocketChannel.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private void startSelector() throws IOException {
        selector = Selector.open();

        serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

        System.out.println("startSelector succ");

        while (true) {

            int readyChannels = selector.selectNow();

            if (readyChannels == 0) {
                continue;
//                    return;
            }

            Iterator<SelectionKey> keyIterator = selector.selectedKeys().iterator();
            while (keyIterator.hasNext()) {
                SelectionKey key = keyIterator.next();
                if (key.isAcceptable()) {
                    // a connection was accepted by a ServerSocketChannel.
                    connectEven(key, selector);
                } else if (key.isConnectable()) {
                    // a connection was established with a remote server.
                    System.out.println("a connection was established with a remote server.");
                } else if (key.isReadable()) {
                    // a channel is ready for reading
                    readData(key);
                } else if (key.isWritable()) {
                    // a channel is ready for writing
                    System.out.println("a channel is ready for writing");
                }
                keyIterator.remove();
            }
        }
    }

    private void startServer() throws Exception {
        //开启服务器
        serverSocketChannel = ServerSocketChannel.open();


        serverSocketChannel.socket().bind(StringUtils.isEmpty(hostname) ?
                new InetSocketAddress(port) : new InetSocketAddress(hostname, port));

        serverSocketChannel.configureBlocking(false);

        System.out.println("startServer succ");

    }


    private void connectEven(SelectionKey key, Selector selector) {
        try {
            ServerSocketChannel server = (ServerSocketChannel) key.channel();
            SocketChannel channel = server.accept();
            channel.configureBlocking(false);
            //获得远程连接的IP地址
            System.out.println("已有连接，连接的ip地址是" + channel.socket().getInetAddress());
            channel.register(selector, SelectionKey.OP_READ);
        } catch (IOException e) {
            e.printStackTrace();
        }

    }

    /**
     * 数据读取处理
     *
     * @param key
     */
    private void readData(SelectionKey key) {
        SocketChannel socketChannel = (SocketChannel) key.channel();
        ByteBuffer buf = ByteBuffer.allocate(1024);
        try {
            int readSize = socketChannel.read(buf);
            if (readSize > 0) {
                byte[] data = buf.array();
                String msg = new String(data, "UTF-8").trim();
                System.out.println(msg);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    public static void main(String[] args) {
        try(NioServer nioServer =  NioServer.port(9999)) {
            nioServer.start();
        }catch (Exception e){
            e.printStackTrace();
        }
    }     
}

Client demo

package nio;

import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SocketChannel;

/**
 * @author zhengyumin
 * @description nio客户端demo
 * @date 2019-09-25 11:36 AM
 */
public class NioClientDemo {
    public static void main(String[] args) throws IOException {

        SocketChannel socketChannel = SocketChannel.open();

        socketChannel.connect(new InetSocketAddress(9999));

        String newData = "New String to write to file..." + System.currentTimeMillis();

        ByteBuffer buf = ByteBuffer.allocate(48);
        buf.clear();
        buf.put(newData.getBytes());

        buf.flip();

        while(buf.hasRemaining()) {
            socketChannel.write(buf);
        }

    }
}

Pipe(todo)

Charset(todo)

File

Path(todo)

Files(todo)

Java NIO vs. IO

When studying both the Java NIO and IO API’s, a question quickly pops into mind:

When should I use IO and when should I use NIO?

Main Differences Betwen Java NIO and IO

The table below summarizes the main differences between Java NIO and IO. I will get into more detail about each difference in the sections following the table.

IO	NIO	Diff
Stream oriented	Buffer oriented	Cache
Blocking IO	Non blocking IO	Threads
	Selectors

How NIO and IO Influences Application Design

Whether you choose NIO or IO as your IO toolkit may impact the following aspects of your application design:

The API calls to the NIO or IO classes.
The processing of data.
The number of thread used to process the data.

The API calls to the NIO or IO classes.

Of course the API calls when using NIO look different than when using IO. This is no surprise. Rather than just read the data byte for byte from e.g. an InputStream, the data must first be read into a buffer, and then be processed from there.

The processing of data.


Java IO: Reading data from a blocking stream.


Java NIO: Reading data from a channel until all needed data is in buffer.

Summary

NIO allows you to manage multiple channels (network connections or files) using only a single (or few) threads, but the cost is that parsing the data might be somewhat more complicated than when reading data from a blocking stream.

If you need to manage thousands of open connections simultanously, which each only send a little data, for instance a chat server, implementing the server in NIO is probably an advantage. Similarly, if you need to keep a lot of open connections to other computers, e.g. in a P2P network, using a single thread to manage all of your outbound connections might be an advantage. This one thread, multiple connections design is illustrated in this diagram:

Java NIO: A single thread managing multiple connections.

Java NIO: A single thread managing multiple connections.

If you have fewer connections with very high bandwidth, sending a lot of data at a time, perhaps a classic IO server implementation might be the best fit. This diagram illustrates a classic IO server design:

Java IO: A classic IO server design - one connection handled by one thread.

Java IO: A classic IO server design - one connection handled by one thread.

转载请注明来源，欢迎对文章中的引用来源进行考证，欢迎指出任何有错误或不够清晰的表达。可以在下面评论区评论，也可以邮件至 951488791@qq.com