Open in app

Sign in

Write

Sign in

Java 學習記錄89 — NIO

張小雄
32 min readDec 7, 2021

--

今天我們要學的是 NIO

首先，先來學習如何操作一般的 txt 檔案

先手動創建一個文檔，待會用 NIO 來讀取跟新增內容

data.txt

Line 1
Line 2
Line 3

Main.java

public class Main {
    public static void main(String[] args) {
        try {
            Path dataPath = Paths.get("src/nonBlockingIO/data.txt");
     
            List<String> lines = Files.readAllLines(dataPath);
            for (String line : lines) {
                System.out.println(line);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

哇賽，這部份比舊 IO 還簡單，還不用手動關閉

輸出結果：

Line 1

Line 2

Line 3

Main.java

public class Main {
    public static void main(String[] args) {
        try {
            Path dataPath = Paths.get("src/nonBlockingIO/data.txt");
            Files.write(dataPath, "\nLine 4".getBytes(StandardCharsets.UTF_8), StandardOpenOption.APPEND);
            List<String> lines = Files.readAllLines(dataPath);
            for (String line : lines) {
                System.out.println(line);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

新增內容也是很簡單

但因為是寫成 byte 所以要轉格式

write 最後一個參數是把新內容從檔案的結尾處新增上去

沒有註明的話，默認是檔案存在的話就蓋掉內容，不存在就新增檔案

若想寫多一點內容，則可以使用 string builder

readAllLines 默認的編碼就是UTF-8

若想要讀別的編碼就在第二個位置新增參數

輸出結果：

Line 1

Line 2

Line 3

Line 4

接下來嘗試寫 binary file

try (FileOutputStream binFile = new FileOutputStream("src/nonBlockingIO/data.dat");
             FileChannel binChannel = binFile.getChannel()) {
            byte[] outputBytes = "Hello World!".getBytes();
            ByteBuffer buffer = ByteBuffer.wrap(outputBytes);
            int numBytes = binChannel.write(buffer);
            System.out.println("numBytes written was: " + numBytes);
        } catch (IOException e) {
            e.printStackTrace();
        }

data.dat

Hello World!

輸出結果：

numBytes written was: 12

簡單的說就是把 buffer 寫進 channel

.dat 裡面的輸出，看起來正常是因為用的是 UTF-8

接下來嘗試寫入數字

try (FileOutputStream binFile = new FileOutputStream("src/nonBlockingIO/data.dat");
             FileChannel binChannel = binFile.getChannel()) {
            byte[] outputBytes = "Hello World!".getBytes();
            ByteBuffer buffer = ByteBuffer.wrap(outputBytes);
            int numBytes = binChannel.write(buffer);
            System.out.println("numBytes written was: " + numBytes);            ByteBuffer intBuffer = ByteBuffer.allocate(Integer.BYTES);
            intBuffer.putInt(666);
            numBytes = binChannel.write(intBuffer);
            System.out.println("numBytes written was: " + numBytes);        } catch (IOException e) {
            e.printStackTrace();
        }

data.dat

Hello World!

輸出結果：

numBytes written was: 12

numBytes written was: 0

為什麼數字沒被寫進去呢？

當創造 buffer 的時候，其位置會被設置到 0

而使用 putInt() 的時候，則會改變其位置，位置會來到 put 內容的最後面

下一行 channel.write() 會去讀 buffer 當前位置，所以沒讀到東西

如果我們要從位置 0 開始讀，則要自己手動設置，調用 flip()

上面 wrap() 不用設置的原因是，因為已經自動幫我們用好了

P.S 若是使用 IO 的話，這些都不用設置，背後都幫我們處理好了

在下方新增調用之後

intBuffer.putInt(666);
intBuffer.flip();

data.dat

Hello World!  �

輸出結果：

numBytes written was: 12

numBytes written was: 4

此時可以在文件內確認數字被寫入了，也可從輸出結果重複確認，int 的確是 4 個 byte

假設我們想繼續寫入數字，這次寫個負數

intBuffer.putInt(666);
intBuffer.flip();
numBytes = binChannel.write(intBuffer);
System.out.println("numBytes written was: " + numBytes);intBuffer.putInt(-123);
numBytes = binChannel.write(intBuffer);
System.out.println("numBytes written was: " + numBytes);

輸出結果：

numBytes written was: 12

numBytes written was: 4

Exception in thread “main” java.nio.BufferOverflowException

at java.base/java.nio.Buffer.nextPutIndex(Buffer.java:674)

at java.base/java.nio.HeapByteBuffer.putInt(HeapByteBuffer.java:413)

at nonBlockingIO.Main.main(Main.java:35)

會發現報錯了

為什麼呢？

因為我們開頭設置過 buffer 的容量就是 int 即 4 個 byte

ByteBuffer intBuffer = ByteBuffer.allocate(Integer.BYTES);

繼續寫入已經超過了限制容量，所以報錯了

解決辦法一樣還是 flip()

intBuffer.flip();
intBuffer.putInt(-123);
intBuffer.flip();
numBytes = binChannel.write(intBuffer);
System.out.println("numBytes written was: " + numBytes);

data.dat

Hello World!  �����

輸出結果：

numBytes written was: 12
numBytes written was: 4
numBytes written was: 4

Q：

why flip before putInt?

A：

lecture248 by Quasar

when you do a putInt on the ByteBuffer, the marker for the ByteBuffer advances by 4 positions (1 for each Byte in the int)…this means when you do a write without flip, the ByteBuffer is read from the 5th position that it is currently at. So, when you do a flip(), it goes back to the 0 position and then you can do a write so that it reads from 0th position.

繼續新增

RandomAccessFile randomAccessFile = new RandomAccessFile("src/nonBlockingIO/data.dat", "rwd");byte[] b = new byte[outputBytes.length];
randomAccessFile.read(b);
System.out.println(new String(b));long int1 = randomAccessFile.readInt();
long int2 = randomAccessFile.readInt();
System.out.println(int1);
System.out.println(int2);

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

Hello World!

123

-789

此處驗證了，寫跟讀可以不用同一個 package

可以用 NIO 寫出 和 IO 讀入，也可以反過來

// read from NIO
RandomAccessFile randomAccessFile = new RandomAccessFile("src/nonBlockingIO/data.dat", "rwd");
FileChannel channel = randomAccessFile.getChannel();
long numBytesRead = channel.read(buffer);
System.out.println("outputBytes = " + new String(outputBytes));

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

outputBytes = Hello World!

把上面用 IO 寫的註釋掉，改用 NIO 來寫

看似一切正常，但其實 read() 沒有發揮作用

因為這邊沒有用 flip()， 而前面寫入後 buffer's postion 已經在尾端了

下方來做驗證

// read from NIO
RandomAccessFile randomAccessFile = new RandomAccessFile("src/nonBlockingIO/data.dat", "rwd");
FileChannel channel = randomAccessFile.getChannel();
outputBytes[0] = 'a';
outputBytes[1] = 'b';
long numBytesRead = channel.read(buffer);
System.out.println("outputBytes = " + new String(outputBytes));

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

outputBytes = abllo World!

可以看到 array 裡的字被改變了

// read from NIO
RandomAccessFile randomAccessFile = new RandomAccessFile("src/nonBlockingIO/data.dat", "rwd");
FileChannel channel = randomAccessFile.getChannel();
outputBytes[0] = 'a';
outputBytes[1] = 'b';
buffer.flip();
long numBytesRead = channel.read(buffer);
System.out.println("outputBytes = " + new String(outputBytes));

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

outputBytes = Hello World!

加上 flip() 後又恢復正常了

P.S 老實講這段我也是霧沙沙，為啥 NIO 變那麼複雜

// read from NIO
RandomAccessFile randomAccessFile = new RandomAccessFile("src/nonBlockingIO/data.dat", "rwd");
FileChannel channel = randomAccessFile.getChannel();
outputBytes[0] = 'a';
outputBytes[1] = 'b';
buffer.flip();
long numBytesRead = channel.read(buffer);
if(buffer.hasArray()){
    System.out.println("outputBytes = " + new String(buffer.array()));
}
intBuffer.flip();
numBytesRead = channel.read(intBuffer);
intBuffer.flip();
System.out.println(intBuffer.getInt());
intBuffer.flip();
numBytesRead = channel.read(intBuffer);
intBuffer.flip();
System.out.println(intBuffer.getInt());
channel.close();
randomAccessFile.close();

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

outputBytes = Hello World!

123

-789

第二種取得 buffer 的方法

//            // Relative read
//            intBuffer.flip();
//            numBytesRead = channel.read(intBuffer);
//            intBuffer.flip();
//            System.out.println(intBuffer.getInt());
//            intBuffer.flip();
//            numBytesRead = channel.read(intBuffer);
//            intBuffer.flip();
//            System.out.println(intBuffer.getInt());            // Absolute read
            intBuffer.flip();
            numBytesRead = channel.read(intBuffer);
            System.out.println(intBuffer.getInt(0));
            intBuffer.flip();
            numBytesRead = channel.read(intBuffer);
            System.out.println(intBuffer.getInt(0));            channel.close();
            randomAccessFile.close();

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

outputBytes = Hello World!

123

-789

指定 buffer 的位置，就可以少用幾次 flip()

// Absolute read
intBuffer.flip();
numBytesRead = channel.read(intBuffer);
System.out.println(intBuffer.getInt(0));
intBuffer.flip();
numBytesRead = channel.read(intBuffer);
intBuffer.flip();
System.out.println(intBuffer.getInt(0));
System.out.println(intBuffer.getInt());

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

outputBytes = Hello World!

123

-789

-789

在 getInt() 加上指定位置不會影響到 buffer 的位置

可以看到最後一行還是順利打印出內容並沒有跳出 exception

Tim 說盡量不要兩者一起混用，像這個案例，不然後面維護會很頭痛

try (FileOutputStream binFile = new FileOutputStream("src/nonBlockingIO/data.dat");
             FileChannel binChannel = binFile.getChannel()) {
            byte[] outputBytes = "Hello World!".getBytes();
//            ByteBuffer buffer = ByteBuffer.wrap(outputBytes);
            ByteBuffer buffer = ByteBuffer.allocate(outputBytes.length);
            buffer.put(outputBytes);

回到上方把 buffer wrap() 改成跟下方數字的寫法一樣

ByteBuffer intBuffer = ByteBuffer.allocate(Integer.BYTES);
intBuffer.putInt(123);

輸出結果：

numBytes written was: 0

numBytes written was: 4

numBytes written was: 4

outputBytes = {����rld!

-789

Exception in thread “main” java.lang.IndexOutOfBoundsException

at java.base/java.nio.Buffer.checkIndex(Buffer.java:693)

at java.base/java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:406)

at nonBlockingIO.Main.main(Main.java:86)

為什麼會報錯呢？

Tim 說：教你一個咒語，在 NIO 遇到問題就用 flip()

buffer.put(outputBytes);
int numBytes = binChannel.write(buffer);

在 NIO 兩種動作交換時，放入跟讀取

就要把 buffer 位置設回到開頭，不然就會超出位置

buffer.put(outputBytes);
buffer.flip();
int numBytes = binChannel.write(buffer);

輸出結果：

numBytes written was: 12

numBytes written was: 4

numBytes written was: 4

outputBytes = Hello World!

123

-789

-789

沒使用 flip() 的話，因為buffer's postion 的位置在最後面，導致繼續寫就超過 allocate() 的大小了，所以報錯。

package nonBlockingIO;import java.io.FileOutputStream;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;public class Main {
    public static void main(String[] args) {        try (FileOutputStream binFile = new FileOutputStream("src/nonBlockingIO/data.dat");
             FileChannel binChannel = binFile.getChannel()) {
            ByteBuffer buffer = ByteBuffer.allocate(100);            byte[] outputBytes = "Hello World!".getBytes();
            buffer.put(outputBytes);
            buffer.putInt(111);
            buffer.putInt(-222);
            byte[] outputByte2 = "Go away!".getBytes();
            buffer.put(outputByte2);
            buffer.putInt(666);
            buffer.putInt(-777);
            buffer.flip();
            binChannel.write(buffer);
            RandomAccessFile ra = new RandomAccessFile("src/nonBlockingIO/data.dat", "rwd");
            FileChannel channel = ra.getChannel();
            ByteBuffer readBuffer = ByteBuffer.allocate(100);
            channel.read(readBuffer);
            readBuffer.flip();            byte[] inputString = new byte[outputBytes.length];
            readBuffer.get(inputString);
            System.out.println("inputString = " + new String(inputString));
            System.out.println("int1 = " + readBuffer.getInt());
            System.out.println("int2 = " + readBuffer.getInt());
            byte[] inputString2 = new byte[outputByte2.length];
            readBuffer.get(inputString2);
            System.out.println("inputString2 = " + new String(inputString2));
            System.out.println("int3 = " + readBuffer.getInt());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

輸出結果：

inputString = Hello World!

int1 = 111

int2 = -222

inputString2 =Go away!

int3 = 666

這邊展示了，把寫出跟讀入，各用單一的 buffer 來完成

//            // unchain
//            byte[] outputBytes = "Hello World!".getBytes();
//            buffer.put(outputBytes);
//            buffer.putInt(111);
//            buffer.putInt(-222);
//            byte[] outputByte2 = "Go away!".getBytes();
//            buffer.put(outputByte2);
//            buffer.putInt(666);
//            buffer.putInt(-777);
//            buffer.flip();
//            binChannel.write(buffer);            // chain
            byte[] outputBytes = "Hello World!".getBytes();
            byte[] outputByte2 = "Go away!".getBytes();
            buffer.put(outputBytes).putInt(111).putInt(-222).put(outputByte2).putInt(666).putInt(-777);
            buffer.flip();
            binChannel.write(buffer);

這邊展示連在一起的寫法

因為 put() 返回的是 bytebuffer所以可以這樣寫

package nonBlockingIO;import java.io.FileOutputStream;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;public class Main {
    public static void main(String[] args) {
        // write part
        try (FileOutputStream binFile = new FileOutputStream("src/nonBlockingIO/data.dat");
             FileChannel binChannel = binFile.getChannel()) {
            ByteBuffer buffer = ByteBuffer.allocate(100);
            byte[] outputBytes = "Hello World!".getBytes();
            buffer.put(outputBytes);
            long int1Pos = outputBytes.length;
            buffer.putInt(111);
            long int2Pos = int1Pos + Integer.BYTES;
            buffer.putInt(-222);
            byte[] outputByte2 = "Go away!".getBytes();
            buffer.put(outputByte2);
            long int3Pos = int2Pos + Integer.BYTES + outputByte2.length;
            buffer.putInt(666);
            buffer.flip();
            binChannel.write(buffer);
            // read part
            RandomAccessFile ra = new RandomAccessFile("src/nonBlockingIO/data.dat", "rwd");
            FileChannel channel = ra.getChannel();
            
            // backward sequence            ByteBuffer readBuffer = ByteBuffer.allocate(Integer.BYTES);
            channel.position(int3Pos);
            channel.read(readBuffer);
            readBuffer.flip();
            System.out.println("int3 = " + readBuffer.getInt());            readBuffer.flip();
            channel.position(int2Pos);
            channel.read(readBuffer);
            readBuffer.flip();
            System.out.println("int2 = " + readBuffer.getInt());            readBuffer.flip();
            channel.position(int1Pos);
            channel.read(readBuffer);
            readBuffer.flip();
            System.out.println("int1 = " + readBuffer.getInt());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

輸出結果：

int3 = 666

int2 = -222

int1 = 111

這邊展示如何倒著順序讀

// write int first, then string
            byte[] outputString = "Hello World!".getBytes();
            long str1Pos = 0;
            long newInt1Pos = outputString.length;
            long newInt2Pos = newInt1Pos + Integer.BYTES;
            byte[] outputString2 = "Go away!".getBytes();
            long str2Pos = newInt2Pos + Integer.BYTES;
            long newInt3Pos = str2Pos + outputString2.length;            ByteBuffer intBuffer = ByteBuffer.allocate(Integer.BYTES);
            intBuffer.putInt(222);
            intBuffer.flip();
            binChannel.position(newInt1Pos);
            binChannel.write(intBuffer);            intBuffer.flip();
            intBuffer.putInt(-987);
            intBuffer.flip();
            binChannel.position(newInt2Pos);
            binChannel.write(intBuffer);            intBuffer.flip();
            intBuffer.putInt(10000);
            intBuffer.flip();
            binChannel.position(newInt3Pos);
            binChannel.write(intBuffer);            binChannel.position(str1Pos);
            binChannel.write(ByteBuffer.wrap(outputString));
            binChannel.position(str2Pos);
            binChannel.write(ByteBuffer.wrap(outputString2));

這邊展示數字跟字符串 分開寫入

RandomAccessFile copyFile = new RandomAccessFile("src/nonBlockingIO/dataCopy.dat", "rw");
FileChannel copyChannel = copyFile.getChannel();
long numTransferred = copyChannel.transferFrom(channel,0, channel.size());
System.out.println("Num transferred = " + numTransferred);channel.close();
ra.close();
copyChannel.close();

輸出結果：

int3 = 666

int2 = -222

int1 = 111

Num transferred = 16

這是複製檔案的功能

但可以看到大小不對

因為是 tansferFrom 用的是相對位置而不是絕對位置

這邊是重複利用 channel 該 buffer's position

因為上方的 channel.read(readBuffer); 已經改變了

RandomAccessFile copyFile = new RandomAccessFile("src/nonBlockingIO/dataCopy.dat", "rw");
FileChannel copyChannel = copyFile.getChannel();
channel.position(0);
long numTransferred = copyChannel.transferFrom(channel,0, channel.size());
System.out.println("Num transferred = " + numTransferred);channel.close();
ra.close();
copyChannel.close();

輸出結果：

int3 = 666

int2 = -222

int1 = 111

Num transferred = 32

把位置設回去就解決了

RandomAccessFile copyFile = new RandomAccessFile("src/nonBlockingIO/dataCopy.dat", "rw");
FileChannel copyChannel = copyFile.getChannel();
channel.position(0);
// channel.position(0);
// long numTransferred = copyChannel.transferFrom(channel, 0, channel.size());
long numTransferred = copyChannel.transferTo(0, channel.size(), copyChannel);
System.out.println("Num transferred = " + numTransferred);channel.close();
ra.close();
copyChannel.close();

輸出結果：

int3 = 666

int2 = -222

int1 = 111

Num transferred = 32

也可以改成用 transferTo

接著介紹 Pipe

try {
            Pipe pipe = Pipe.open();            Runnable writer = new Runnable() {
                @Override
                public void run() {
                    try {
                        SinkChannel sinkChannel = pipe.sink();
                        ByteBuffer buffer = ByteBuffer.allocate(56);                        for (int i = 0; i < 10; i++) {
                            String currentTime = "The time is: " + System.currentTimeMillis();                            buffer.put(currentTime.getBytes());
                            buffer.flip();                            while (buffer.hasRemaining()) {
                                sinkChannel.write(buffer);
                            }
                            buffer.flip();
                            Thread.sleep(100);
                        }                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            };
        } catch (IOException e) {
            e.printStackTrace();
        }

這是寫入的部份

// introduce pipe        try {
            Pipe pipe = Pipe.open();            Runnable writer = new Runnable() {
                @Override
                public void run() {
                    try {
                        SinkChannel sinkChannel = pipe.sink();
                        ByteBuffer buffer = ByteBuffer.allocate(56);                        for (int i = 0; i < 10; i++) {
                            String currentTime = "The time is: " + System.currentTimeMillis();                            buffer.put(currentTime.getBytes());
                            buffer.flip();                            while (buffer.hasRemaining()) {
                                sinkChannel.write(buffer);
                            }
                            buffer.flip();
                            Thread.sleep(100);
                        }                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            };            Runnable reader = new Runnable() {
                @Override
                public void run() {
                    try {
                        Pipe.SourceChannel sourceChannel = pipe.source();
                        ByteBuffer buffer = ByteBuffer.allocate(56);                        for (int i = 0; i < 10; i++) {
                            int byteRead = sourceChannel.read(buffer);
                            ;
                            byte[] timeString = new byte[byteRead];
                            buffer.flip();
                            buffer.get(timeString);
                            System.out.println("Reader Thread: " + new String(timeString));
                            buffer.flip();
                            Thread.sleep(100);
                        }
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            };
            
            new Thread(writer).start();
            new Thread(reader).start();                    } catch (IOException e) {
            e.printStackTrace();
        }

加入讀取跟開始的部份

輸出結果：

Reader Thread: The time is: 1636117448501

Reader Thread: The time is: 1636117448633

Reader Thread: The time is: 1636117448744

Reader Thread: The time is: 1636117448853

Reader Thread: The time is: 1636117448970

Reader Thread: The time is: 1636117449086

Reader Thread: The time is: 1636117449200

Reader Thread: The time is: 1636117449303

Reader Thread: The time is: 1636117449413

Reader Thread: The time is: 1636117449523

這邊展示了如何用 Pipe 在不同 Thread 傳資料

上述代碼都紀錄在我的 Github

Java
Programming
Learning
Coding
Taiwan

--

--

張小雄
張小雄

Written by 張小雄

35 Followers
19 Following

記錄成為軟體工程師的過程

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams