C:使用fread()/ fgets()而不是fgetc()逐行读取文本文件(具有可变长度的行)(块I / O与字符I / O)

C: Reading a text file (with variable-length lines) line-by-line using fread()/fgets() instead of fgetc() (block I/O vs. character I/O)

是否有一个getline函数使用fread(块I / O)而不是fgetc(字符I / O)?

通过fgetc逐个字符读取文件会降低性能。我们认为,为了提高性能,我们可以在getline的内部循环中通过fread使用块读取。但是,这会导致读取超出行尾的潜在不良影响。至少,这需要实现getline来跟踪文件的"未读"部分,这需要超出ANSI C FILE语义的抽象。这不是我们想要实现的东西!

我们已经分析了我们的应用程序,并且性能下降的原因在于我们通过fgetc逐字符地消耗大文件。相比之下,其余的间接费用实际上是微不足道的。我们始终从头到尾顺序读取文件的每一行,并且可以在读取期间锁定整个文件。这可能会使基于freadgetline易于实现。

那么,是否存在使用fread(块I / O)而不是fgetc(字符I / O)的getline函数?我们很确定这样做,但是如果没有,我们应该如何实现呢?

更新找到了Paul Hsieh撰写的有用的文章,《用C处理用户输入》。这是基于fgetc的方法,但是对替代方法进行了有趣的讨论(从gets的严重程度开始,然后讨论fgets):

On the other hand the common retort from C programmers (even those considered experienced) is to say that fgets() should be used as an alternative. Of course, by itself, fgets() doesn't really handle user input per se. Besides having a bizarre string termination condition (upon encountering
or EOF, but not \0) the mechanism chosen for termination when the buffer has reached capacity is to simply abruptly halt the fgets() operation and \0 terminate it. So if user input exceeds the length of the preallocated buffer, fgets() returns a partial result. To deal with this programmers have a couple choices; 1) simply deal with truncated user input (there is no way to feed back to the user that the input has been truncated, while they are providing input) 2) Simulate a growable character array and fill it in with successive calls to fgets(). The first solution, is almost always a very poor solution for variable length user input because the buffer will inevitably be too large most of the time because its trying to capture too many ordinary cases, and too small for unusual cases. The second solution is fine except that it can be complicated to implement correctly. Neither deals with fgets' odd behavior with respect to '\0'.

Exercise left to the reader: In order to determine how many bytes was really read by a call to fgets(), one might try by scanning, just as it does, for a '
' and skip over any '\0' while not exceeding the size passed to fgets(). Explain why this is insufficient for the very last line of a stream. What weakness of ftell() prevents it from addressing this problem completely?

Exercise left to the reader: Solve the problem determining the length of the data consumed by fgets() by overwriting the entire buffer with a non-zero value between each call to fgets().

So with fgets() we are left with the choice of writing a lot of code and living with a line termination condition which is inconsistent with the rest of the C library, or having an arbitrary cut-off. If this is not good enough, then what are we left with? scanf() mixes parsing with reading in a way that cannot be separated, and fread() will read past the end of the string. In short, the C library leaves us with nothing. We are forced to roll our own based on top of fgetc() directly. So lets give it a shot.

那么,是否存在基于fgetsgetline函数(并且不会截断输入)?


不要使用fread。使用fgets。我认为这是一个家庭作业/班级项目问题,所以我没有提供完整的答案,但是如果您说不是,我会提供更多建议。绝对有可能仅使用fgets提供100%的GNU样式getline语义,包括嵌入的空字节,但是这需要一些聪明的思考。

确定,请更新,因为这不是家庭作业:

  • memset您的缓冲区到'
    '
  • 使用fgets
  • 使用memchr查找第一个'
    '
  • 如果未找到'
    '
    ,则该行比缓冲区长。扩大缓冲区,用'
    '
    填充新部分,然后将fgets填充到新部分,必要时重复进行。
  • 如果'
    '
    之后的字符是'\0',则fgets由于到达行尾而终止。
  • 否则,fgets由于到达EOF而终止,'
    '
    从您的memset遗留下来,前一个字符是fgets写入的终止空值,而在此之前的字符是读取的实际数据的最后一个字符。

如果您不关心支持带有嵌入式null的行,则可以消除memset并使用strlen代替memchr(无论哪种方式,该null都不会终止读取;它只是您读取的一部分,排队)。

还有一种方法可以对fscanf"%123[^
]"
说明符(其中123是您的缓冲区限制)执行相同的操作,这使您可以灵活地停止非换行符(ala GNU getdelim)。但是,除非您的系统具有非常精美的scanf实现,否则它可能会很慢。


fgets和fgetc / setvbuf之间没有太大的性能差异。
尝试:

1
2
3
4
5
6
7
8
9
10
11
int c;
FILE *f = fopen("blah.txt","r");
setvbuf(f,NULL,_IOLBF,4096); /* !!! check other values for last parameter in your OS */
while( (c=fgetc(f))!=EOF )
{
  if( c=='
'
)
    ...
  else
    ...
}