Path: csiph.com!xmission!news.snarked.org!news.linkpendium.com!news.linkpendium.com!panix!usenet.stanford.edu!not-for-mail From: dirk+bash@testssl.sh Newsgroups: gnu.bash.bug Subject: Re: bash sockets: printf \x0a does TCP fragmentation Date: Sat, 22 Sep 2018 11:50:17 +0200 Lines: 70 Approved: bug-bash@gnu.org Message-ID: References: <20180921231101307758654@bob.proulx.com> NNTP-Posting-Host: lists.gnu.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: usenet.stanford.edu 1537609827 17336 208.118.235.17 (22 Sep 2018 09:50:27 GMT) X-Complaints-To: action@cs.stanford.edu Cc: bug-bash@gnu.org To: Bob Proulx Envelope-to: bug-bash@gnu.org Openpgp: preference=signencrypt Autocrypt: addr=dirk+bash@testssl.sh; prefer-encrypt=mutual; keydata= xsFNBFSxqIwBEAClqHueTe+Ro+I4jReXss4DKwfeKhl23yuEZ7wN7GxBwGxslYxY15sJWhJ1 C0eglGwNGd/P3ObGgdNiT/DDvQzKFe8wcpCUnAOuE+ZnylBnqVD6xUmd+mPl6j9B7ByP42mY 81EK2ZSJ84mWjwOjT66pxjvWq7jzWfBA+QEQTlxiF18CFiHnv9XoLAD5yk04x6DyBGQZtobB YmN9uujK+nrbXPO4qQ+h4xWhWZ5U/77O2R0JgvrwvROfa+sS+oaP+9TTAko9BJYr0wfZ1meY C/fqidB0ihgTJjHgR4wjeklA5xvrwMHNSNcCN+fPYzfUcQsrQ+kY+NskkPYNr/3zbAvhPoT4 YXk8XQs1pNFNS9qF33iKWgU+zqKTC4NxqduVZGinpGhDZcACIE2fXrllMB7NW6/9BundIoaf XRchVPxJpujvQE91IAQktSWqVbQb7O8CSJQybDoAPRZvh/9ayIhBRx76oYLNAIrjFElBEhx9 hAloH0wCcXS1DvbvcQL9qz61qF09fuH/T1jB/YRkzk1EMmB3pO4hjC27yNiUuDKZlz5PUZoZ O7SJFdcvDxlAmBK6I3D1TQvldDljbZl/vhcuSmX41rp//ChTVxdE7p7RnK2KxoZLob2m8AYO zva5bokelFHQFjFz28/FPiMLmns/rsnZWRcQA4dCUSyHPnlRTQARAQABzSFEaXJrIFdldHRl ciA8ZGlyay53ZXR0ZXJAZ3V1Zy5kZT7CwX0EEwECACcCGwMFCQlmAYACHgECF4AFAlSxqlAF CwkIBwMFFQoJCAsFFgIDAQAACgkQya7s4dCnRWnq4Q//WV0ebAlej0vMVpp6H8SlnHvPwMAe ZuUDOSxUjnlk9QUtxlkjrnBRbpwTYJaLBi6dk8AE53qRFUOEKu7KMS2PenDDU1WcRGddkFu+ YW/WIzE+pplMhPnk3F13uOJhO8DDFoxQdgHbRPFlcfCQaLoN8gYMPAaSHnLndNTUvhSQQYmu obFbR2QQDbtM3TxO80fpxQm3gSyHi303abODYCF7k0yw57IY1lczsPUv1PYEM+9Y9TW7s9JS mKvO4wQvBdUXaghhoR13qWrxbiPq12UxtB3tHkS6tRcalQqxT7ZWsktqqvha30fueB+8Otlh GOhA6sFSWpegw2iOoC6L+9JYsxx9wYBtWFM39tDh/sBcz3YKMpM2XdasHsLfy5rE1El4N6WP wGCQ+iKFZt6CtO+LCI3XZPN1mqfx93zOA288nnqPMCkMfqyTMAeKzsuoFgbzfDy1TQBy+09w QVY9d+JrrIL4YO2UoVfsm2YjcsKjjKzsmxUrCxKfcO0CCEdh7+LjmtHanFi2/Oy+jt49rikW 2Gt+lrwZZFJh3v05+pVvENpJn3VTNQkGz0dif7yNaFQUYyTwQ5ot1B51FRlAUE6KETfO3Zbc nRzSxQ8jqs42u04ZVgaTLlRCPooyhR3ULUGIUOxxo1HitDioVUT1HPyGxniqtEexIGJEVn3h EwCLpw7OwU0EVLGojAEQAK3c39f4dV5ZM0jdfIwF+oJXZr2mfof7jMDFvZKOmjb2S6fqQnVi 47rr8n9kcGxlWmfsBVOOekVa6BCOJvJVY+dzaqJA9Y0h/0RGCmVqgIx/dSgUFtRQ/HnWeQ2V o6hpOBhE+XNLqaZZSPuZ+kpjppXMxMh0GZyGpUWUgA3HFPbZl3YcZk3hAU0ZlJZ9qy2NZYYy V6WQylO8sCSJ+rb4pBiHAE+BsIM6YbU9CER1zGty5sd9QWoS33/YWrbXuC5dSiF6xbncrVE4 74cGvke+z3VewST8S7S+xy0Xsh+qbgFOK+6qWQJmKxj19/J//+JoDOoTTPW1izJc5NSxuO9o Tls4TD+njA2JK/A0Jnlve+/vW83lkbd4iJhr1hDDPReyl5CIkdlaXJsu0e9j77wen6qfgRRs AjvVgE/kox235bms6MLcHgH2tiO613I8iu+weL3xsBJXpUB0V8eHwusA9JOuMGLeWrQFt9dh glfeoOLNzt1eA23v0ncHHF7kdL5yOTbkLtryWx6ztjnmWKV1+zKh0Z3NR25t0s+gR8Mjz09m Qp9fKVsTQCcloQuTl/Z6DwORMkhtxEam39OSvrIbjQF3tB4qEKSxauX+pMo3l8tEAx7FoT2F xCPKkr+OX3CVSBRF/aP1IErc9r82JdErMmaPM/35LPE7ApSaRRfc8cdPABEBAAHCwWUEGAEC AA8FAlSxqIwCGwwFCQlmAYAACgkQya7s4dCnRWm1iw/9E17eZ2VfFlTd5Vnh6e2nqxRO3Z74 pIvAKuKE2o2LmJvncZDNUgVktaCiA7H328bnUSd1+Hx+O/0UshcqQbeSGBq4BO51mUnbdBVV XtF1P6h7Jcxwv5/UEWQ1g21jUAflZnQTAFSLHvWP4s80Shsj5vzljFN2wutZXHN7mdMZBqxI d8HLUn5ECChHUB5a6uqHosp4UpMQJTTmbaA56KcqgFRG36TKYtemNmnhvXYhGxR+02cPmJs1 qaMbbkNBAQWLJjb2ix5oPmAccDleEdLxZVYCNX4Cq5IBiqdG988vdq8kyjTcFjw74mT4wNBS xhr6P6ox+eublYRiYP2MHHEaH64plj0XIXoaXEUK7pB02ATguDc+iMq6GUBbSF3nExtLGVUk Id3XjVoJdqz1XJ25z3MhPBAnr07eA9PUIzpWGJpCBBkqV4rCL26VZZgyDzFoEs69f3PzsjZr cqVIaXmlDy/ij1OY0pExMT9qa/AXRN1PSLgOvYqzkcwM1Xo9DYycJa1viSagPrLdmYBGQBaM xFmTNHuJYbTwXaZC18YkXJSlXbsUazoki+4WLUF/DPY2yL7fdvNhe1SPzWPSKQspv2b453KH A/EJRgHEoqVEYSm/zyLD9oXcS50KlwgghDPnFLp0yp/ooRFPy4wQ3j0xp9pZVV2V7b+i081m A/hqo7fCwfMEGAEIACYWIQQzLjFaOt2q7moROVfJruzh0KdFaQUCWUP5wgIbAgUJAuY8gACB CRDJruzh0KdFaXYgBBkWCAAdFiEEu+44xYV6RNw/MO5qzFXuFSFkehkFAllD+cIACgkQzFXu FSFkehlnFgEAzMMahul6aHTiMK1EPvBJkqUtZXhpCPNW713G58DkonEBAKxaRFItRxf/p1E+ 9aAINhfXLP1DvSuWP/GFiAda+sEPH/MP/1Y5Qa+WBs4AWNXyF7DPsc45z0E2R38k4VC9+hve 2nEav/Joa0oqQJlJAME/bgXX1YQ9O6i/kAzIbeFp+WMTMU/dP/K2PTh0E8tH/bM1Rv9oDLoO 9TFJx/3roBvym9oTJuWVYgfd/57JABn1VKiyP60ZEKdTlelH5uohkhEJ87G1kAOOKAE/kDfx 6UZW8SvMKESNyHSvg+H7YpJhuMBoMhj5rjYjTwbM2gcH73c8QPan2T354NwULQsQ1ISjT1R+ nj0EDq+nQGHOUZnKs2jvFn+qwV7pHSChd5N+D7AkJ0PbkXKfrgp7mDPOkK+02h0m35iPUiGD X823M/342k812BCK69VeXKXmRX6L2wThb4LKzpDKMLtVk8UzJ+Kpb/uua+SToPWxjnhkWouL 66gqKaDQfqE3S3D/CunoPzt3zZaFhqfAiCYunpbHHlgqhUCeOGe+/frT297DAkS+W4gGSwrT HMtS3FMs5uLjT+e6zizZ6oWjqz7xc9KCGvSbbxOR4Xh7V7whDkEVh4RtUx/MORGnkQ/8Vlkl PorqZV8FgMB+tMrWDcljdpHJ39lWyLQQPmwqhr8huF6wM6egsRmpYTA+tnkAPRzOqEZDsenT y9FhQVoiW4+yZcMAtBAxVjrF7IPwZ8ZiC+MIUtpCTezO9lA2AmQVOvAO+rLMMmTeWOjJwsF8 BBgBCAAmFiEEMy4xWjrdqu5qETlXya7s4dCnRWkFAllD+eYCGwwFCQLmPIAACgkQya7s4dCn RWnJtA/9FjnVZxJz60YZOSE9ouWKdbZaX9M0n5q3hWtt6YIJAvbTNvA5lZbBxwvp4MuJPlhL OzkAdTfVe95OxpnEzALeT7/RA8s38ir67MQQBK/EEImyeTGu/UL8kT57lsW/T15lFGD30ixD 8W3NBfUS/ohF8DPNRtSeQx9WUdXXSwR/O/SNtSo7+M4xqmoA9xcYmPGc8l8nxDfvvqtX6nLe wWqg4m5fKxVbl1DD/xdz7P2sec/BeIJwDZxAVw4i8tQlgWY5wnUmHbf8rA788aNfMTmRwwZz 4PtqKy8P9WLPMf6ogqACRbnqJEcVrO32bHe7jiSo8b4Eq0dj6lQSLxx+jBCZjCVoHe+kjP0M gsEjW5smwBFj5Gx3pbk3YaeIQxxo6Aw8ktguyNuTmpgMbz3f8lUULqruiltOI442Hg/6YBxe ZUPNEKVpttqXIGzi+LHVeLFZ7d5acpCAYWsyg5ycUUbvM/bpbimOx9l8LJrbytdHAkWllpBA eqB64BWtQrlBwXS5yFYNgzlvJXGsbGaFHzIi5AKjYPbPw8BzF1JKe48yJQg0O3L9HJOwAsPg 0znJ14tpiW56CDABH2Yt7w0FZ32FEFpUiPN8jRt5QLQ5quAAltH69RkzQ2thIOCBiEdXBIph QDq64ekD2YgbnJXCsfli5AOhc8aH7yQnOmVG6LcMuqw= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.0 In-Reply-To: <20180921231101307758654@bob.proulx.com> Content-Language: en-US X-Df-Sender: NDM2MjM5 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 80.67.31.101 X-BeenThere: bug-bash@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Bug reports for the GNU Bourne Again SHell List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Xref: csiph.com gnu.bash.bug:14632 On 9/22/18 7:30 AM, Bob Proulx wrote: > dirk+bash@testssl.sh wrote: >> we discovered a strange phenomenon in the project testssl.sh: > > You are doing something that is quite unusual. You are using a shell > script direction on a TCP socket. That isn't very common. Do you think there should be a paragraph NOT COMMON where bash sockets should rather belong to? > More > typically one would use a C program instead. So it isn't surprising > that you are finding interactions that are not well known. Bob, my intention was not to discuss program languages and what is typical with you or anybody else here. >> printf -- "$data" >&5 2>/dev/null > > Why is stderr discarded? That is almost always bad because it > discards any errors that might occur. You probably shouldn't do this.> > What happens if $data contains % format strings? What happens if the > format contains a sequence such as \c? This looks problematic. This > is not a safe programming proctice. I doubt you can judge on this by just looking at a single line of code -- the project has > 18k LoC in bash. Github is the place to discuss and do PRs for our project. >> If there's a workaround, please let me know. (tried to add "%b" with no >> effect). Otherwise I believe it's a bug. > > You can re-block the output stream using other tools such as 'cat' or > 'dd'. Since you are concerned about block size then perhaps dd is the > better of the two. > > | cat cat has a problem with binary chars, right? And: see below. > Or probably better: > > | dd status=none bs=1M > > Or use whatever block size you wish. The 'dd' program will read the > input into its buffer and then output that block of data all in one > write(2). That seems to be what you are wanting. We actually use dd to read from the socket. Of course we could use writing to it as well -- at a certain point of time. Still, a prerequisite would be that printf is the culprit and not how bash + libs do sockets. > P.S. You can possibly use the 'stdbuf' command to control the output > buffering depending upon the program. > > info stdbuf That could be an option, thanks. Need to check though whether a) it doesn't fragment then -- not sure while reading it b) it's per default available on every platform supported by testssl.sh. Cheers, Dirk