mirror of
git://git.sv.gnu.org/coreutils.git
synced 2026-06-16 23:23:43 +02:00
Pádraig Brady
38cf97c1ff
The default memory allocation with pipes was too passive/static, resulting in not allocating enough memory to enable threading. By dynamically reallocating the buffer when reading from unknown sized inputs we better use available memory and threads. $ time seq 10000000 -1 0 | sort-old >/dev/null real 0m16.523s user 0m16.900s sys 0m0.167s $ time seq 10000000 -1 0 | sort-old -S1G >/dev/null real 0m12.263s user 0m29.646s sys 0m0.527s $ time seq 10000000 -1 0 | sort-new >/dev/null real 0m12.994s user 0m31.266s sys 0m0.716s It also avoids the overhead of writing to temp files for modestly sized inputs. For example the following input would induce interaction with temp storage: $ seq 125000 | wc -c 763895 * src/sort.c (sort_buffer_size): Rename to ... (sort_buffer_policy): ... here, and adjust to set an initial size and limit, rather than just a size. (fillbuf): Add a POLICY parameter, and use that to call maybe_growbuf() as needed. (maybe_growbuf): Return true if POLICY dictates we should grow the buffer, and try_growbuf() was able to reallocate the larger buffer. * tests/sort/sort-buffer-size.sh: Add a new test. * tests/local.mk: Reference new test. * NEWS: Mention the improvement. Related to https://bugs.gnu.org/10877