Path: csiph.com!goblin1!goblin.stu.neva.ru!usenet.stanford.edu!not-for-mail From: L A Walsh Newsgroups: gnu.bash.bug Subject: Re: [PATCH 5.1] zread: read files in 4k chunks Date: Tue, 23 Jun 2020 12:29:42 -0700 Lines: 30 Approved: bug-bash@gnu.org Message-ID: References: <20200622055328.155106-1-Jason@zx2c4.com> <3cac1f66-697d-396d-3dc8-fc5b621638a4@case.edu> <271ebec9-fe1f-7823-a236-2e734657a279@iki.fi> NNTP-Posting-Host: lists.gnu.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: usenet.stanford.edu 1592940602 13571 209.51.188.17 (23 Jun 2020 19:30:02 GMT) X-Complaints-To: action@cs.stanford.edu Cc: Ilkka Virta , bug-bash@gnu.org, Chester Ramey To: "Jason A. Donenfeld" Envelope-to: bug-bash@gnu.org X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6yL4xU4l2HhZXOurlg2ZRV+mKYY/ihxsll2RPFGS6AI=; b=aJ1tGP47tEfMmkJX7Wlv6bKzPXAFFH4mAjeO7ejhy1DDR/WVDYg8qaobjez0eBTL1b 41ktTgE7VlJ8QGJrMAWRk3iN+Phia5aRuPXOMaW2rk1YSwi9Y/MRATE78dpNjvtewvDR oQ57eIzMXLCTFeeitrLQonk/pUm3djRnSw48Bxma2lhiYidKu44M1hE9+lNy8tY53K56 PpY09OL/92IOpulr6GR3y6RBI5fX5s2OGHVtVHcB3rv10tK/E+hifxw1XrdOKdhfEDSV 0cRFkpV0sABp0BMNF+xhWAgOzgpMss1qo+5m6rvTCqAFAru6aXChZb4Xm0IuRU4q/hWw x29Q== X-Gm-Message-State: AOAM531WW1C6ZrTvk/1WLR5b+NGsurq6Db7ve4EaMYnOXh+nmq5JPdGM B7ZTypqL8sB096K4NG4qq3mz/yTZfwaRzvWXBoCsJauK9Og= X-Google-Smtp-Source: ABdhPJwmw6h+28u+2bhM0pDnokukC5yOiB5/7wv0klSWCxG8GVkh62ZFCDPXFXYHKoWt7jgBt2+F2Y3G/i8HrsncE8E= X-Received: by 2002:a6b:780d:: with SMTP id j13mr27963892iom.66.1592940596865; Tue, 23 Jun 2020 12:29:56 -0700 (PDT) In-Reply-To: Received-SPF: softfail client-ip=209.85.166.67; envelope-from=gsuite@tlinx.org; helo=mail-io1-f67.google.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/23 15:29:57 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=_AUTOLEARN X-Spam_action: no action X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: bug-bash@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Bug reports for the GNU Bourne Again SHell List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Mailman-Original-Message-ID: X-Mailman-Original-References: <20200622055328.155106-1-Jason@zx2c4.com> <3cac1f66-697d-396d-3dc8-fc5b621638a4@case.edu> <271ebec9-fe1f-7823-a236-2e734657a279@iki.fi> Xref: csiph.com gnu.bash.bug:16445 The 'stat(2)' system call returns an optimal i/o size for files since some files in addition to being on disks with a 4k sector size (making 128bytes a slow choice for reads, and a real slow choice for writes), also can be on a RAID with it's own optimal i/o size. I think the 'stat(1)' prog shows the same value with "%o": "stat -c%o" . 65536 On Mon, Jun 22, 2020 at 1:19 PM Jason A. Donenfeld wrote: > On Mon, Jun 22, 2020 at 2:16 PM Ilkka Virta wrote: > > > > On 22.6. 19.35, Chet Ramey wrote: > > > On 6/22/20 1:53 AM, Jason A. Donenfeld wrote: > > >> Currently a static sized buffer is used for reading files. At the > moment > > >> it is extremely small, making parsing of large files extremely slow. > > >> Increase this to 4k for improved performance. > > > > > > I bumped it up to 1024 initially for testing. > > > > It always struck me as odd that Bash used such a small read of 128 > > bytes. Most of the GNU utils I've looked at on Debian use 8192, and a > > simple test program seems to indicate glibc's stdio reads 4096 bytes at > > one read() call. > > Plus most other shells people use... > >