如何使用Java逐行读取大型文本文件？

Question

如何使用Java逐行读取大型文本文件？

我需要使用Java逐行读取5-6GB左右的大型文本文件。

我怎么能这么快做呢？

Answer 1

一个常见的模式是使用try&nbsp;(BufferedReader&nbsp;br&nbsp;=&nbsp;new&nbsp;BufferedReader(new&nbsp;FileReader(file)))&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;String&nbsp;line; &nbsp;&nbsp;&nbsp;&nbsp;while&nbsp;((line&nbsp;=&nbsp;br.readLine())&nbsp;!=&nbsp;null)&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;process&nbsp;the&nbsp;line. &nbsp;&nbsp;&nbsp;&nbsp;}}如果假设没有字符编码，则可以更快地读取数据。ASCI-7，但不会有太大的区别。你对数据的处理很可能需要更长的时间。编辑：一种较不常见的模式，它避免了line漏水。try(BufferedReader&nbsp;br&nbsp;=&nbsp;new&nbsp;BufferedReader(new&nbsp;FileReader(file)))&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;for(String&nbsp;line;&nbsp;(line&nbsp;=&nbsp;br.readLine())&nbsp;!=&nbsp;null;&nbsp;)&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;process&nbsp;the&nbsp;line. &nbsp;&nbsp;&nbsp;&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;line&nbsp;is&nbsp;not&nbsp;visible&nbsp;here.}更新：在Java 8中，您可以做到try&nbsp;(Stream<String>&nbsp;stream&nbsp;=&nbsp;Files.lines(Paths.get(fileName)))&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;stream.forEach(System.out::println);}注意：您必须将流放在试着用资源块，以确保对它调用#CLOSE方法，否则基础文件句柄在GC很久之后才会关闭。

Answer 2

可以指定缓冲区大小，也可以使用默认大小。对于大多数用途，缺省值都足够大。// Open the fileFileInputStream fstream = new FileInputStream("textfile.txt"); BufferedReader br = new BufferedReader(new InputStreamReader(fstream));String strLine; //Read File Line By Linewhile ((strLine = br.readLine()) != null)   {   // Print the content on the console   System.out.println (strLine);}//Close the input streamfstream.close();

Answer 3

一次爪哇-8发布(2014年3月)，您将能够使用流：try&nbsp;(Stream<String>&nbsp;lines&nbsp;=&nbsp;Files.lines(Paths.get(filename),&nbsp;Charset.defaultCharset()))&nbsp;{ &nbsp;&nbsp;lines.forEachOrdered(line&nbsp;->&nbsp;process(line));}打印文件中的所有行：try&nbsp;(Stream<String>&nbsp;lines&nbsp;=&nbsp;Files.lines(file,&nbsp;Charset.defaultCharset()))&nbsp;{ &nbsp;&nbsp;lines.forEachOrdered(System.out::println);}

Answer 4

下面是一个包含完整错误处理和支持前Java 7字符集规范的示例。使用Java 7，您可以使用具有资源的尝试语法，这使代码更加简洁。如果您只想要默认的字符集，可以跳过InputStream并使用FileReader。InputStream&nbsp;ins&nbsp;=&nbsp;null;&nbsp;//&nbsp;raw&nbsp;byte-streamReader&nbsp;r&nbsp;=&nbsp;null;&nbsp;//&nbsp;cooked&nbsp;readerBufferedReader&nbsp;br&nbsp;=&nbsp;null;&nbsp;//&nbsp;buffered&nbsp;for&nbsp;readLine()try&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;String&nbsp;s; &nbsp;&nbsp;&nbsp;&nbsp;ins&nbsp;=&nbsp;new&nbsp;FileInputStream("textfile.txt"); &nbsp;&nbsp;&nbsp;&nbsp;r&nbsp;=&nbsp;new&nbsp;InputStreamReader(ins,&nbsp;"UTF-8");&nbsp;//&nbsp;leave&nbsp;charset&nbsp;out&nbsp;for&nbsp;default &nbsp;&nbsp;&nbsp;&nbsp;br&nbsp;=&nbsp;new&nbsp;BufferedReader(r); &nbsp;&nbsp;&nbsp;&nbsp;while&nbsp;((s&nbsp;=&nbsp;br.readLine())&nbsp;!=&nbsp;null)&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;System.out.println(s); &nbsp;&nbsp;&nbsp;&nbsp;}}catch&nbsp;(Exception&nbsp;e){ &nbsp;&nbsp;&nbsp;&nbsp;System.err.println(e.getMessage());&nbsp;//&nbsp;handle&nbsp;exception}finally&nbsp;{ &nbsp;&nbsp;&nbsp;&nbsp;if&nbsp;(br&nbsp;!=&nbsp;null)&nbsp;{&nbsp;try&nbsp;{&nbsp;br.close();&nbsp;}&nbsp;catch(Throwable&nbsp;t)&nbsp;{&nbsp;/*&nbsp;ensure&nbsp;close&nbsp;happens&nbsp;*/&nbsp;}&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;if&nbsp;(r&nbsp;!=&nbsp;null)&nbsp;{&nbsp;try&nbsp;{&nbsp;r.close();&nbsp;}&nbsp;catch(Throwable&nbsp;t)&nbsp;{&nbsp;/*&nbsp;ensure&nbsp;close&nbsp;happens&nbsp;*/&nbsp;}&nbsp;} &nbsp;&nbsp;&nbsp;&nbsp;if&nbsp;(ins&nbsp;!=&nbsp;null)&nbsp;{&nbsp;try&nbsp;{&nbsp;ins.close();&nbsp;}&nbsp;catch(Throwable&nbsp;t)&nbsp;{&nbsp;/*&nbsp;ensure&nbsp;close&nbsp;happens&nbsp;*/&nbsp;}&nbsp;}}以下是Groovy版本，具有完全错误处理：File&nbsp;f&nbsp;=&nbsp;new&nbsp;File("textfile.txt");f.withReader("UTF-8")&nbsp;{&nbsp;br&nbsp;-> &nbsp;&nbsp;&nbsp;&nbsp;br.eachLine&nbsp;{&nbsp;line&nbsp;-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;println&nbsp;line; &nbsp;&nbsp;&nbsp;&nbsp;}}