Path: csiph.com!xmission!news.snarked.org!news.linkpendium.com!news.linkpendium.com!panix!usenet.stanford.edu!not-for-mail From: Bob Proulx Newsgroups: gnu.bash.bug Subject: Re: bash sockets: printf \x0a does TCP fragmentation Date: Sat, 22 Sep 2018 23:51:08 -0600 Lines: 76 Approved: bug-bash@gnu.org Message-ID: References: <20180922111950901701520@bob.proulx.com> <20180921231101307758654@bob.proulx.com> <714e1ba0-0052-2f2b-676d-778f2b7129c1@testssl.sh> <7769.1537667711@jinx.noi.kre.to> NNTP-Posting-Host: lists.gnu.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: usenet.stanford.edu 1537681881 25416 208.118.235.17 (23 Sep 2018 05:51:21 GMT) X-Complaints-To: action@cs.stanford.edu To: bug-bash@gnu.org Envelope-to: bug-bash@gnu.org Mail-Followup-To: bug-bash@gnu.org Content-Disposition: inline In-Reply-To: <7769.1537667711@jinx.noi.kre.to> User-Agent: Mutt/1.10.1 (2018-07-13) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 96.88.95.61 X-BeenThere: bug-bash@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Bug reports for the GNU Bourne Again SHell List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Xref: csiph.com gnu.bash.bug:14642 Robert Elz wrote: > ps: f there was actually a desire to use dd to do re-buffering, the > correct usage is not to use "bs=" (which simply does read write with > a buffer that size) but "obs=" which reads (using ibs if needed, which it > would not be here), copies to an output buffer and writes only when that > buffer is full (or on EOF on the input). If the goal is to minimize writes then it won't matter as long as the buffer size picked is larger than needed. Using the same buffer size for input and output is usually most efficient. First let's set it to something small to prove that it is buffering as expected. $ printf -- "%s\n" one two | strace -o /tmp/out -e read,write dd status=none bs=2 ; cat /tmp/out one two ... read(0, "on", 2) = 2 write(1, "on", 2) = 2 read(0, "e\n", 2) = 2 write(1, "e\n", 2) = 2 read(0, "tw", 2) = 2 write(1, "tw", 2) = 2 read(0, "o\n", 2) = 2 write(1, "o\n", 2) = 2 read(0, "", 2) = 0 +++ exited with 0 +++ Lots of reads and writes but all as expected. Or set just the output buffer size large. Then the input buffer size defaults to 512 bytes on my system. $ printf -- "%s\n" one two | strace -o /tmp/out -e write,read dd status=none obs=1M ; cat /tmp/out one two ... read(0, "one\ntwo\n", 512) = 8 read(0, "", 512) = 0 write(1, "one\ntwo\n", 8) = 8 +++ exited with 0 +++ But even if ibs is much too small it still behaves okay with a small input buffer size and a large output buffer size. $ printf -- "%s\n" one two | strace -o /tmp/out -e write,read dd status=none ibs=2 obs=1M ; cat /tmp/out one two ... read(0, "on", 2) = 2 read(0, "e\n", 2) = 2 read(0, "tw", 2) = 2 read(0, "o\n", 2) = 2 read(0, "", 2) = 0 write(1, "one\ntwo\n", 8) = 8 +++ exited with 0 +++ Then set both ibs and obs to be something quite large using bs= and let it gather up all of the input and write with that buffer size. $ printf -- "%s\n" one two | strace -o /tmp/out -e write,read dd status=none bs=1M ; cat /tmp/out one two ... read(0, "one\ntwo\n", 1048576) = 8 write(1, "one\ntwo\n", 8) = 8 read(0, "", 1048576) = 0 +++ exited with 0 +++ It seems to me that using a large buffer size for both read and write would be the most efficient. It can then use the same buffer that data was read into for the output buffer directly. Bob