Processing files is a common operation in java, and a little bit of speed improvement in operating a "large file" (for example, more than 64M) will bring about a huge performance improvement. However, we often use BufferxStream to deal with large files directly, often unable to do so.

The "Mapping File Access" mechanism in java solves this problem. It puts the smaller part of the large file in memory first and the rest of the data to be read in the hard disk. But we can use this large file as a very large array through such a mechanism. Is it very similar to how applications are used in operating systems? Yes, in fact, java uses the file mapping tools of the underlying operating system to maximize performance.

In order to be both readable and writable, we use Random Access File to test, pass through the same channel on the file, and then call MappedByteBuffer generated by map() in FileChannel. In the map method, there are three parameters map (FileChannel. MapMode, long position, long size).

Where mode determines the read/write mechanism for mapped file access, it can be set MapMode.READ_ONLY， MapMode.READ_WRITE，MapMode.PRIVATE They can be seen by their names, with special attention to MapMode.PRIVATE Method, how do you operate improperly will cause the files you operate can not be shared.

The position must be a non-negative number to begin with the location of the file mapping in the region

Size is a mapping for the size of the region; it must be a non-negative number, not greater than Integer.MAX_VALUE

In fact, the setting of position and size means that we map the smaller part of a large file.

Now let's look at the performance improvement when we use Mapped File Access.

The following is the code section:

  1 package com.company;
  2 
  3 import java.io.*;
  4 import java.nio.IntBuffer;
  5 import java.nio.channels.FileChannel;
  6 
  7 /**
  8  * Created by chunmiao on 17-3-15.
  9  */
 10 public class MappedIOTest {
 11     //Number of input values
 12     private static int numOfInts = 4000000;
 13     //Test the number of read and write values
 14     private static int numOfUbuffInts = 200000;
 15 
 16     private abstract static class Tester{
 17         private String name;
 18         public Tester(String name){
 19             this.name = name;
 20         }
 21 
 22         //Test timing
 23         public void runTest(){
 24             System.out.println(name + ": ");
 25             try {
 26                 long start = System.nanoTime();
 27                 test();
 28                 long runtime = System.nanoTime() - start;
 29                 //output test()Running time to determine the size of operation time
 30                 System.out.format("%.2f\n",runtime/1.0e9);
 31             }catch (IOException e){
 32                 throw new RuntimeException(e);
 33             }
 34         }
 35 
 36         public abstract void test() throws IOException;
 37     }
 38 
 39     private static Tester[] tests = {
 40             //Write data directly
 41             new Tester("Stream Write") {
 42                 @Override
 43                 public void test() throws IOException {
 44                     DataOutputStream dos = new DataOutputStream(
 45                             new BufferedOutputStream(
 46                                     new FileOutputStream(new File("temp.tmp"))));
 47                     for (int i = 0; i < numOfInts; i ++){
 48                         dos.writeInt(i);
 49                     }
 50                     dos.close();
 51                 }
 52             },
 53             //Use MappedByteBuffer Write files
 54             new Tester("Mapped Write") {
 55                 @Override
 56                 public void test() throws IOException {
 57                     FileChannel fc = new RandomAccessFile("temp.tmp","rw").getChannel();
 58                     //fc.size()The return is byte The number of units.
 59                     IntBuffer ib = fc.map(FileChannel.MapMode.READ_WRITE,0,fc.size()).asIntBuffer();
 60                     for(int i =0 ; i < numOfInts ; i ++){
 61                         try {
 62                             ib.put(i);
 63                         }catch (Exception e){
 64                             System.out.println(i);
 65                         }
 66                     }
 67                     fc.close();
 68                 }
 69             },
 70             //Direct reading of data
 71             new Tester("Stream Read") {
 72                 @Override
 73                 public void test() throws IOException {
 74                     DataInputStream dis = new DataInputStream(
 75                             new BufferedInputStream(new FileInputStream("temp.tmp")));
 76                     for (int i = 0 ; i < numOfInts ; i ++){
 77                         dis.readInt();
 78                     }
 79                     dis.close();
 80                 }
 81             },
 82             //Use MappedByteBuffer Read files
 83             new Tester("Mapped Read") {
 84                 @Override
 85                 public void test() throws IOException {
 86                     FileChannel fc = new FileInputStream(new File("temp.tmp")).getChannel();
 87                     IntBuffer ib = fc.map(FileChannel.MapMode.READ_ONLY,0,fc.size()).asIntBuffer();
 88                     while (ib.hasRemaining()){
 89                         ib.get();
 90                     }
 91                     fc.close();
 92                 }
 93             },
 94             //Read directly/Write operation
 95             new Tester("Stream Read/Write") {
 96                 @Override
 97                 public void test() throws IOException {
 98                     RandomAccessFile raf = new RandomAccessFile(new File("temp.tmp"),"rw");
 99                     //Misalignment of data for simultaneous reading and writing
100                     raf.writeInt(1);
101                     for (int i = 0 ; i < numOfUbuffInts ; i ++){
102                         //The reason for the reduction of 4 is raf.length()Return to byte For unit space size, and 1( int) = 4 (byte),So we need to subtract 4
103                         raf.seek(raf.length() - 4);
104                         raf.writeInt(raf.readInt());
105                     }
106                     raf.close();
107                 }
108             },
109             //Use MappedByteBuffer Read/Write operation
110             new Tester("Mapped Read/Write") {
111                 @Override
112                 public void test() throws IOException {
113                     FileChannel fc = new RandomAccessFile(new File("temp.tmp"),"rw").getChannel();
114                     IntBuffer ib = fc.map(FileChannel.MapMode.READ_WRITE,0,fc.size()).asIntBuffer();
115                     ib.put(0);
116                     for (int i = 1 ; i < numOfUbuffInts ; i ++){
117                         //Misalignment of data for simultaneous reading and writing
118                         ib.put(ib.get(i - 1));
119                     }
120                     fc.close();
121                 }
122             }
123     };
124 
125     public static void main(String[] args) {
126         //Testing
127         for (Tester tester : tests){
128             tester.runTest();
129         }
130     }
131 }

The following are the results of the operation: