Path: csiph.com!xmission!news.snarked.org!news.linkpendium.com!news.linkpendium.com!panix!usenet.stanford.edu!not-for-mail From: Chet Ramey Newsgroups: gnu.bash.bug Subject: Re: [PATCH 5.1] zread: read files in 4k chunks Date: Mon, 22 Jun 2020 17:07:27 -0400 Organization: ITS, Case Western Reserve University Lines: 22 Approved: bug-bash@gnu.org Message-ID: References: <20200622055328.155106-1-Jason@zx2c4.com> <3cac1f66-697d-396d-3dc8-fc5b621638a4@case.edu> <271ebec9-fe1f-7823-a236-2e734657a279@iki.fi> <9994508a-78f5-cac9-053c-df504f459c73@case.edu> Reply-To: chet.ramey@case.edu NNTP-Posting-Host: lists.gnu.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Trace: usenet.stanford.edu 1592860057 22718 209.51.188.17 (22 Jun 2020 21:07:37 GMT) X-Complaints-To: action@cs.stanford.edu Cc: chet.ramey@case.edu To: Ilkka Virta , "Jason A. Donenfeld" , bug-bash@gnu.org Envelope-to: bug-bash@gnu.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=case.edu; s=smtp-primary; t=1592860052; bh=TFL0kJDnZqRlcEJtCPc26nUNl5dfWNUsjJ1nzJbRUd0=; h=Reply-To:Cc:Subject:To:References:From:Message-ID:Date: MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=bvTbJkrJL9bstJYutejTwpg5dQc4fK4c+RxDOd2X8pOc84a8yrt2lZRNtpAab3WjrU 68HQZOTzLaGrQUwvA9jBTz3bs0QOq4lZG6Vle2o1Dbk3cYFlAqzLC6lQUi/23TLqPCq jW2ZzIHQyOPRf7rtOUdtkXJwGhKVzWdO+zwkzTJt1bHJKe903P5qpRacAVv8a/bwG8I aQ0L81oc8x5NcohBnPWOH949iQIa1Z3P6YXqWeIm15MfLJR+u8J3W7LXi71wLxWJ1ip UQmF5o4WXebm1JqLHIul/XQarvedll7idhrmio06Gw5v4wRiaMZYfQB46KxQwpRhk1I MgHByz/g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=case.edu; s=smtp-primary; t=1592860050; bh=kGYx0/aIK3DkLxMjENgClAdj6J66z3tZJaHNTFxn3tw=; h=Reply-To:Cc:Subject:To:References:From:Message-ID:Date: MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=fKECYzsiMPM8ZXvdwQuAAjBlRCb7NVm8eNoGTLw5g70gIcXg011jqnWW0OeKgHy/xq HpQjGIcjPwflRM/Qy44hwjPOsoth5dHx6ZzXmc+ClHwkQxFGogHZteZzGY0r5bdEC3j pF7uGCSVoZAcmf8aQnU1pCf4Ua9YNE7PFjHKirme/wi1msmRjF/niww7/h60YnhKH4U +CU8590hOi5bNlHA3kKDoV5PBnjugtifCHOt+oZnLYnXMa1bHbUFy8t7zRsRL3KOpKu OM68NY3hqDruFoRlPnF9kS8oEcxtyp4Y4PVqQPiGT3yJzyFV8cAD8NhFgV40YSKnJ+G e7e6YagA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=case.edu; s=g-case; h=reply-to:cc:subject:to:references:from:autocrypt:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=kGYx0/aIK3DkLxMjENgClAdj6J66z3tZJaHNTFxn3tw=; b=YYUiUVKkrp7thtXNvz0H/5Hja1w5VVzaFZbVCU7LcIqVGOVDBaecatPcyuRCtNJI5j yCib9dlaMnWoi1wUqR8wXbSHRIckMm8Guq0HyqcinBlCQ9QMqSUNjIS3v6qT9pkCPYFV gQhJE4pzqj8gWjN3K93ejq+Vw49Sant/MilO5kid/kh8mQSxO/2aDx0iY0hQLsrYihu9 Bo5oA06ASKMuLb2qkgs2CIrYEVnnZX1hfoySRIe6ZE1cnnFrke4xTTxwaMnvj+2LxMwI lLbu31Sx2XXB6ioCkDry57lSpnL6HhGxZBtpddiExu7F7l0VFfoTn8otsXc/4Uxk6VWG GZyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:cc:subject:to:references:from:autocrypt :organization:message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=kGYx0/aIK3DkLxMjENgClAdj6J66z3tZJaHNTFxn3tw=; b=JLM+9GrR0xB6xqLNik2pqCf63FKsnodWjeJJDHiQTbBusOnlQpMTowfeq/GuU9kU4m em2yWdqVfNl/Jybb3eSMfIw4ZJuTGUUj5PWYZhMPHnkawjQvibqSpXEulhDQi4Zlqa28 jviRBgm/vNkaeVTQad5Be8+R5RONdu/7Pyx18odcW0CkqKj7MmNE4WmfL2pgAvdLvWQk ZuYui9STujEqj6E3ERDCi4Ng0IUmGXtzaMot7gYsiKJ8//llYLlN2xAYFv+8I8UJtk+I YaQQFEaCCHrpbc7z68l3nL4fHEHRC3F+ClHIjzKG13wIG5zqBNq/QglqxUxkdfJ2fCXr dwHQ== X-Gm-Message-State: AOAM533ga0hogsiwRjlc67wo5+Wz5TiAa100tzNEwiwSYRAf75YrsKz4 I6qxziyNCKnVkgvMPDGSDL8BNUyfzAsVbccstSi6hoqnkb6Vo8UD2+4ZLTW3mM0H8SGXeL61Ux0 zc6J8bzAJITk= X-Received: by 2002:ac8:3908:: with SMTP id s8mr11665696qtb.97.1592860049647; Mon, 22 Jun 2020 14:07:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzdEYQhboLjS6ApvCsrLbhTTDPeDkXKUaM7QqFjhWTDKnNrSnOrwnUQBXsS+uO1VoIH7b4+hw== X-Received: by 2002:ac8:3908:: with SMTP id s8mr11665671qtb.97.1592860049310; Mon, 22 Jun 2020 14:07:29 -0700 (PDT) Autocrypt: addr=chet.ramey@case.edu; prefer-encrypt=mutual; keydata= mQGiBEEOsGwRBACFa0A1oa71HSZLWxAx0svXzhOZNQZOzqHmSuGOG92jIpQpr8DpvgRh40Yp AwdcXb8QG1J5yGAKeevNE1zCFaA725vGSdHUyypHouV0xoWwukYO6qlyyX+2BZU+okBUqoWQ koWxiYaCSfzB2Ln7pmdys1fJhcgBKf3VjWCjd2XJTwCgoFJOwyBFJdugjfwjSoRSwDOIMf0D /iQKqlWhIO1LGpMrGX0il0/x4zj0NAcSwAk7LaPZbN4UPjn5pqGEHBlf1+xDDQCkAoZ/VqES GZragl4VqJfxBr29Ag0UDvNbUbXoxQsARdero1M8GiAIRc50hj7HXFoERwenbNDJL86GPLAQ OTGOCa4W2o29nFfFjQrsrrYHzVtyA/9oyKvTeEMJ7NA3VJdWcmn7gOu0FxEmSNhSoV1T4vP2 1Wf7f5niCCRKQLNyUy0wEApQi4tSysdz+AbgAc0b/bHYVzIf2uO2lIEZQNNt+3g2bmXgloWm W5fsm/di50Gm1l1Na63d3RZ00SeFQos6WEwLUHEB0yp6KXluXLLIZitEJLQwQ2hldCBSYW1l eSAoQ2FzZSBzdGFuZGFyZCkgPGNoZXQucmFtZXlAY2FzZS5lZHU+iF8EExECAB8FAkPi19EC GwMHCwkIBwMCAQMVAgMDFgIBAh4BAheAAAoJELtYafBk6nSrelkAn31Gsuib7GcCZHbv5L5t VKYR9LklAJ4hzUHKA49Z0QXR+qCb80osIcmPSbkBDQRBDrBvEAQAkK6TAOKBEM+EC4j6V/7o /riVZqcgU5cid2qG9TXdwNtD9a3kvA/ObZBO93sX59wc6Bnwo4VJxsOmMlpGrAjJsxNwg3QH akEtf8LXRbVpj5xStdmBdQZUhIQyalo/2/TZq5OijtddUQcL5cs70hTv/FpT3wUvr2Xr8rjF 41IFEz8AAwcD/A0CZEGlzIrT5WCBnl6xBog/8vKiUCbarByat3d1mL6DbizvKNXQRTC9E/vE dENAWCQCjr75Bu55xT8n3SXGtWdDC5xmZ/P3OBYORP8yl8H8I1FIosWOFirbIeYdZPq8SPD1 HL+EXo9zSiHVrrZRJ19ooCKKbSdXHFCY+aJG+0KZiEkEGBECAAkFAkEOsG8CGwwACgkQu1hp 8GTqdKvjcACfZlkVCDwaz/NTO9cy3t69oWpVPNwAnRwe0qk/WL/gfhH346xh5B3HFbFN User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 In-Reply-To: <271ebec9-fe1f-7823-a236-2e734657a279@iki.fi> Content-Language: en-US X-Mirapoint-IP-Reputation: reputation=Good-1, source=Queried, refid=tid=0001.0A020301.5EF11D4A.000E, actions=tag X-Mirapoint-IP-Reputation: reputation=good-1, source=Fixed, refid=n/a, actions=tag X-Junkmail-Status: score=7/80, host=mpv4-2015.case.edu X-Junkmail-PrAS-Raw: score=7/80, refid=2.7.2:2020.6.22.203316:17:7.944, ip=, rules=__YOUTUBE_RCVD, DKIM_SIGNATURE, __X_GOOGLE_DKIM_SIGNATURE, __HAS_REPLYTO, __HAS_CC_HDR, __SUBJ_REPLY, __BOUNCE_CHALLENGE_SUBJ, __BOUNCE_NDR_SUBJ_EXEMPT, __SUBJ_ALPHA_END, __TO_MALFORMED_2, __MULTIPLE_RCPTS_TO_X2, __TO_NAME, __TO_NAME_DIFF_FROM_ACC, __HAS_REFERENCES, __REFERENCES, __HAS_FROM, FROM_EDU_TLD, __HAS_MSGID, __SANE_MSGID, DATE_TZ_NA, __USER_AGENT, __MOZILLA_USER_AGENT, __MIME_VERSION, __IN_REP_TO, __CT, __CT_TEXT_PLAIN, __CTE, __REPLYTO_SAMEAS_FROM_ADDY, __REPLYTO_SAMEAS_FROM_ACC, __FROM_DOMAIN_IN_ANY_CC1, __FROM_DOMAIN_IN_ANY_CC2, __REPLYTO_SAMEAS_FROM_DOMAIN, __DKIM_ALIGNS_1, __DKIM_ALIGNS_2, __ANY_URI, __URI_MAILTO, __URI_WITH_PATH, __URI_ENDS_IN_SLASH, __URI_NO_WWW, __CP_URI_IN_BODY, __FRAUD_MONEY_BIG_COIN_DIG, __SUBJ_ALPHA_NEGATE, __URI_IN_BODY, __URI_NOT_IMG, __MAIL_CHAIN, __FORWARDED_MSG, __BODY_NO_MAILTO, __NO_HTML_TAG_RAW, [TRUNCATED], so=2010-03-03 19:42:08, dmn=2016-08-03-0138 Received-SPF: pass client-ip=129.22.103.195; envelope-from=chet.ramey@case.edu; helo=mpv4-2015.case.edu X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/22 17:07:30 X-ACL-Warn: Detected OS = Linux 2.4.x-2.6.x [generic] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: bug-bash@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Bug reports for the GNU Bourne Again SHell List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Mailman-Original-Message-ID: <9994508a-78f5-cac9-053c-df504f459c73@case.edu> X-Mailman-Original-References: <20200622055328.155106-1-Jason@zx2c4.com> <3cac1f66-697d-396d-3dc8-fc5b621638a4@case.edu> <271ebec9-fe1f-7823-a236-2e734657a279@iki.fi> Xref: csiph.com gnu.bash.bug:16436 On 6/22/20 4:16 PM, Ilkka Virta wrote: > On 22.6. 19.35, Chet Ramey wrote: >> On 6/22/20 1:53 AM, Jason A. Donenfeld wrote: >>> Currently a static sized buffer is used for reading files. At the moment >>> it is extremely small, making parsing of large files extremely slow. >>> Increase this to 4k for improved performance. >> >> I bumped it up to 1024 initially for testing. > > It always struck me as odd that Bash used such a small read of 128 bytes. > Most of the GNU utils I've looked at on Debian use 8192, and a simple test > program seems to indicate glibc's stdio reads 4096 bytes at one read() call. Yes, 128 is too small for modern systems. It made more sense when the code was written. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet@case.edu http://tiswww.cwru.edu/~chet/