Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1474699 > unrolled thread

Block-level access

Back to article view | Back to linux.kernel

Contents

  Block-level access Alex Austin <aaustin@oceantechonline.com> - 2016-09-01 23:50 +0200
    Re: Block-level access Alex Austin <aaustin@oceantechonline.com> - 2016-09-01 23:50 +0200
      Re: Block-level access Alex Austin <aaustin@oceantechonline.com> - 2016-09-02 16:30 +0200

#1474699 — Block-level access

Hello,
What is the most performant way to directly interface with an attached hard
drive? I've so far used read()/write() on /dev/sd_ but I find error handling
exceedingly difficult, as I don't always even get errors reported, even if the
open() call includes O_DIRECT. I've also used ioctl(SG_IO), but find that it's
extremely slow due to the lack of queuing support in the API. Is there a
mid-level API that will get me decent error handling while allowing command
queuing, or do I just need to make multiple threads all doing separate SG_IO
ioctls?

Thank you,
- Alex Austin

-- 
Intelligence is knowing that a tomato is a fruit; wisdom is knowing that it
doesn't go in a fruit salad; charisma is selling a tomato-based fruit salad.

[toc] | [next] | [standalone]

#1474715

CC'ing linux-scsi and linux-block.

Also, please CC me in replies.

On Thu, Sep 1, 2016 at 4:43 PM, Alex Austin <aaustin@oceantechonline.com> wrote:
> Hello,
> What is the most performant way to directly interface with an attached hard
> drive? I've so far used read()/write() on /dev/sd_ but I find error handling
> exceedingly difficult, as I don't always even get errors reported, even if the
> open() call includes O_DIRECT. I've also used ioctl(SG_IO), but find that it's
> extremely slow due to the lack of queuing support in the API. Is there a
> mid-level API that will get me decent error handling while allowing command
> queuing, or do I just need to make multiple threads all doing separate SG_IO
> ioctls?
>
> Thank you,
> - Alex Austin
>
> --
> Intelligence is knowing that a tomato is a fruit; wisdom is knowing that it
> doesn't go in a fruit salad; charisma is selling a tomato-based fruit salad.



-- 
Intelligence is knowing that a tomoato is a fruit; wisdom is knowing
that it doesn't go in a fruit salad; charisma is selling a
tomato-based fruit salad.

[toc] | [prev] | [next] | [standalone]

#1475195

My access is almost purely sequential and primarily writing, so read-ahead
doesn't help me. What's problematic with pread/pwrite is the lack of error
channel from media errors.

BSG looks very interesting. I'll look further into that today.

On Thu, Sep 1, 2016 at 5:16 PM, Bart Van Assche
<bart.vanassche@sandisk.com> wrote:
> On 09/01/2016 02:48 PM, Alex Austin wrote:
>>
>> CC'ing linux-scsi and linux-block.
>>
>> Also, please CC me in replies.
>>
>> On Thu, Sep 1, 2016 at 4:43 PM, Alex Austin <aaustin@oceantechonline.com>
>> wrote:
>>>
>>> Hello,
>>> What is the most performant way to directly interface with an attached
>>> hard
>>> drive? I've so far used read()/write() on /dev/sd_ but I find error
>>> handling
>>> exceedingly difficult, as I don't always even get errors reported, even
>>> if the
>>> open() call includes O_DIRECT. I've also used ioctl(SG_IO), but find that
>>> it's
>>> extremely slow due to the lack of queuing support in the API. Is there a
>>> mid-level API that will get me decent error handling while allowing
>>> command
>>> queuing, or do I just need to make multiple threads all doing separate
>>> SG_IO
>>> ioctls?
>
>
> What the most efficient way is to interface is with a hard drive depends on
> the I/O pattern. Are you aware that buffered I/O performs read-ahead? Have
> you already tried to tune the read-ahead setting (blockdev --getra /
> blockdev --setra)?
>
> BTW, if you need an example of how to queue SG_IO, you are welcome to have a
> look at the fio source code. You will probably be interested in the bsg API.
> See e.g. https://lwn.net/Articles/174469/.
>
> Bart.



-- 
Intelligence is knowing that a tomoato is a fruit; wisdom is knowing
that it doesn't go in a fruit salad; charisma is selling a
tomato-based fruit salad.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web