Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1674105 > unrolled thread

[PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

Back to article view | Back to linux.kernel

Contents

  [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op Jaegeuk Kim <jaegeuk@kernel.org> - 2017-06-24 18:30 +0200
    [PATCH 2/2] f2fs: report # of free inodes more precisely Jaegeuk Kim <jaegeuk@kernel.org> - 2017-06-24 18:30 +0200
      Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more  precisely Chao Yu <yuchao0@huawei.com> - 2017-06-26 13:00 +0200
        Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more  precisely Jaegeuk Kim <jaegeuk@kernel.org> - 2017-06-26 17:00 +0200
          Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more  precisely Chao Yu <chao@kernel.org> - 2017-06-28 15:10 +0200
            Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more  precisely Jaegeuk Kim <jaegeuk@kernel.org> - 2017-07-01 09:30 +0200
              Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more  precisely Chao Yu <chao@kernel.org> - 2017-07-01 10:40 +0200
    Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order  of page and lock_op Chao Yu <yuchao0@huawei.com> - 2017-06-26 10:20 +0200
      Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order  of page and lock_op Jaegeuk Kim <jaegeuk@kernel.org> - 2017-06-26 17:00 +0200
        Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order  of page and lock_op Chao Yu <chao@kernel.org> - 2017-06-26 17:50 +0200
          Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order  of page and lock_op Jaegeuk Kim <jaegeuk@kernel.org> - 2017-07-01 09:30 +0200
            Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order  of page and lock_op Chao Yu <chao@kernel.org> - 2017-07-01 10:50 +0200
              Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order  of page and lock_op Jaegeuk Kim <jaegeuk@kernel.org> - 2017-07-01 16:30 +0200

#1674105 — [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

- punch_hole
 - fill_zero
  - f2fs_lock_op
  - get_new_data_page
   - lock_page

- f2fs_write_data_pages
 - lock_page
 - do_write_data_page
  - f2fs_lock_op

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
---
 fs/f2fs/data.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 7d3af48d34a9..9141bd19a902 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -1404,8 +1404,9 @@ int do_write_data_page(struct f2fs_io_info *fio)
 		}
 	}
 
-	if (fio->need_lock == LOCK_REQ)
-		f2fs_lock_op(fio->sbi);
+	/* Deadlock due to between page->lock and f2fs_lock_op */
+	if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
+		return -EAGAIN;
 
 	err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
 	if (err)
-- 
2.13.0.rc1.294.g07d810a77f-goog

[toc] | [next] | [standalone]

#1674109 — [PATCH 2/2] f2fs: report # of free inodes more precisely

If the partition is small, we don't need to report total # of inodes including
hidden free nodes.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
---
 fs/f2fs/super.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 8e39b850bfc0..3da6fb276f8b 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -680,6 +680,7 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
 	struct f2fs_sb_info *sbi = F2FS_SB(sb);
 	u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
 	block_t total_count, user_block_count, start_count, ovp_count;
+	u64 avail_node_count;
 
 	total_count = le64_to_cpu(sbi->raw_super->block_count);
 	user_block_count = sbi->user_block_count;
@@ -692,9 +693,16 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
 	buf->f_bfree = user_block_count - valid_user_blocks(sbi) + ovp_count;
 	buf->f_bavail = user_block_count - valid_user_blocks(sbi);
 
-	buf->f_files = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
-	buf->f_ffree = min(buf->f_files - valid_node_count(sbi),
-							buf->f_bavail);
+	avail_node_count = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
+
+	if (avail_node_count > user_block_count) {
+		buf->f_files = user_block_count;
+		buf->f_ffree = buf->f_bavail;
+	} else {
+		buf->f_files = avail_node_count;
+		buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
+					buf->f_bavail);
+	}
 
 	buf->f_namelen = F2FS_NAME_LEN;
 	buf->f_fsid.val[0] = (u32)id;
-- 
2.13.0.rc1.294.g07d810a77f-goog

[toc] | [prev] | [next] | [standalone]

#1674634 — Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more precisely

Hi Jaegeuk,

On 2017/6/25 0:25, Jaegeuk Kim wrote:
> If the partition is small, we don't need to report total # of inodes including
> hidden free nodes.
> 
> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> ---
>  fs/f2fs/super.c | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> index 8e39b850bfc0..3da6fb276f8b 100644
> --- a/fs/f2fs/super.c
> +++ b/fs/f2fs/super.c
> @@ -680,6 +680,7 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
>  	struct f2fs_sb_info *sbi = F2FS_SB(sb);
>  	u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
>  	block_t total_count, user_block_count, start_count, ovp_count;
> +	u64 avail_node_count;
>  
>  	total_count = le64_to_cpu(sbi->raw_super->block_count);
>  	user_block_count = sbi->user_block_count;
> @@ -692,9 +693,16 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
>  	buf->f_bfree = user_block_count - valid_user_blocks(sbi) + ovp_count;
>  	buf->f_bavail = user_block_count - valid_user_blocks(sbi);
>  
> -	buf->f_files = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
> -	buf->f_ffree = min(buf->f_files - valid_node_count(sbi),
> -							buf->f_bavail);
> +	avail_node_count = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
> +
> +	if (avail_node_count > user_block_count) {
> +		buf->f_files = user_block_count;
> +		buf->f_ffree = buf->f_bavail;

f_ffree is limited both by remained free nid count and free block count, so it
needs to change like this?

if (avail_node_count > user_block_count)
	avail_node_count = user_block_count;

buf->f_files = avail_node_count;
buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
					buf->f_bavail);

Thanks,

> +	} else {
> +		buf->f_files = avail_node_count;
> +		buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
> +					buf->f_bavail);
> +	}
>  
>  	buf->f_namelen = F2FS_NAME_LEN;
>  	buf->f_fsid.val[0] = (u32)id;
> 

[toc] | [prev] | [next] | [standalone]

#1674839 — Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more precisely

On 06/26, Chao Yu wrote:
> Hi Jaegeuk,
> 
> On 2017/6/25 0:25, Jaegeuk Kim wrote:
> > If the partition is small, we don't need to report total # of inodes including
> > hidden free nodes.
> > 
> > Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> > ---
> >  fs/f2fs/super.c | 14 +++++++++++---
> >  1 file changed, 11 insertions(+), 3 deletions(-)
> > 
> > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> > index 8e39b850bfc0..3da6fb276f8b 100644
> > --- a/fs/f2fs/super.c
> > +++ b/fs/f2fs/super.c
> > @@ -680,6 +680,7 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
> >  	struct f2fs_sb_info *sbi = F2FS_SB(sb);
> >  	u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
> >  	block_t total_count, user_block_count, start_count, ovp_count;
> > +	u64 avail_node_count;
> >  
> >  	total_count = le64_to_cpu(sbi->raw_super->block_count);
> >  	user_block_count = sbi->user_block_count;
> > @@ -692,9 +693,16 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
> >  	buf->f_bfree = user_block_count - valid_user_blocks(sbi) + ovp_count;
> >  	buf->f_bavail = user_block_count - valid_user_blocks(sbi);
> >  
> > -	buf->f_files = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
> > -	buf->f_ffree = min(buf->f_files - valid_node_count(sbi),
> > -							buf->f_bavail);
> > +	avail_node_count = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
> > +
> > +	if (avail_node_count > user_block_count) {
> > +		buf->f_files = user_block_count;
> > +		buf->f_ffree = buf->f_bavail;
> 
> f_ffree is limited both by remained free nid count and free block count, so it
> needs to change like this?

I thought both of them are same, since node block will consume user block. So,
we don't need to do min() again.

> 
> if (avail_node_count > user_block_count)
> 	avail_node_count = user_block_count;
> 
> buf->f_files = avail_node_count;
> buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
> 					buf->f_bavail);
> 
> Thanks,
> 
> > +	} else {
> > +		buf->f_files = avail_node_count;
> > +		buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
> > +					buf->f_bavail);
> > +	}
> >  
> >  	buf->f_namelen = F2FS_NAME_LEN;
> >  	buf->f_fsid.val[0] = (u32)id;
> > 

[toc] | [prev] | [next] | [standalone]

#1676643 — Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more precisely

On 2017/6/26 22:58, Jaegeuk Kim wrote:
> On 06/26, Chao Yu wrote:
>> Hi Jaegeuk,
>>
>> On 2017/6/25 0:25, Jaegeuk Kim wrote:
>>> If the partition is small, we don't need to report total # of inodes including
>>> hidden free nodes.
>>>
>>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
>>> ---
>>>  fs/f2fs/super.c | 14 +++++++++++---
>>>  1 file changed, 11 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
>>> index 8e39b850bfc0..3da6fb276f8b 100644
>>> --- a/fs/f2fs/super.c
>>> +++ b/fs/f2fs/super.c
>>> @@ -680,6 +680,7 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
>>>  	struct f2fs_sb_info *sbi = F2FS_SB(sb);
>>>  	u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
>>>  	block_t total_count, user_block_count, start_count, ovp_count;
>>> +	u64 avail_node_count;
>>>  
>>>  	total_count = le64_to_cpu(sbi->raw_super->block_count);
>>>  	user_block_count = sbi->user_block_count;
>>> @@ -692,9 +693,16 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
>>>  	buf->f_bfree = user_block_count - valid_user_blocks(sbi) + ovp_count;
>>>  	buf->f_bavail = user_block_count - valid_user_blocks(sbi);
>>>  
>>> -	buf->f_files = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
>>> -	buf->f_ffree = min(buf->f_files - valid_node_count(sbi),
>>> -							buf->f_bavail);
>>> +	avail_node_count = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
>>> +
>>> +	if (avail_node_count > user_block_count) {
>>> +		buf->f_files = user_block_count;
>>> +		buf->f_ffree = buf->f_bavail;
>>
>> f_ffree is limited both by remained free nid count and free block count, so it
>> needs to change like this?
> 
> I thought both of them are same, since node block will consume user block. So,
> we don't need to do min() again.

avail_node_count comes from total free nid counts which is limited with
nid_bitmap size, buf->f_bavail comes from total user block count which
can both cosumed by node and data. So the value of them may not be the same.

Thanks,

> 
>>
>> if (avail_node_count > user_block_count)
>> 	avail_node_count = user_block_count;
>>
>> buf->f_files = avail_node_count;
>> buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
>> 					buf->f_bavail);
>>
>> Thanks,
>>
>>> +	} else {
>>> +		buf->f_files = avail_node_count;
>>> +		buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
>>> +					buf->f_bavail);
>>> +	}
>>>  
>>>  	buf->f_namelen = F2FS_NAME_LEN;
>>>  	buf->f_fsid.val[0] = (u32)id;
>>>
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> 

[toc] | [prev] | [next] | [standalone]

#1679277 — Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more precisely

On 06/28, Chao Yu wrote:
> On 2017/6/26 22:58, Jaegeuk Kim wrote:
> > On 06/26, Chao Yu wrote:
> >> Hi Jaegeuk,
> >>
> >> On 2017/6/25 0:25, Jaegeuk Kim wrote:
> >>> If the partition is small, we don't need to report total # of inodes including
> >>> hidden free nodes.
> >>>
> >>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> >>> ---
> >>>  fs/f2fs/super.c | 14 +++++++++++---
> >>>  1 file changed, 11 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> >>> index 8e39b850bfc0..3da6fb276f8b 100644
> >>> --- a/fs/f2fs/super.c
> >>> +++ b/fs/f2fs/super.c
> >>> @@ -680,6 +680,7 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
> >>>  	struct f2fs_sb_info *sbi = F2FS_SB(sb);
> >>>  	u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
> >>>  	block_t total_count, user_block_count, start_count, ovp_count;
> >>> +	u64 avail_node_count;
> >>>  
> >>>  	total_count = le64_to_cpu(sbi->raw_super->block_count);
> >>>  	user_block_count = sbi->user_block_count;
> >>> @@ -692,9 +693,16 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
> >>>  	buf->f_bfree = user_block_count - valid_user_blocks(sbi) + ovp_count;
> >>>  	buf->f_bavail = user_block_count - valid_user_blocks(sbi);
> >>>  
> >>> -	buf->f_files = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
> >>> -	buf->f_ffree = min(buf->f_files - valid_node_count(sbi),
> >>> -							buf->f_bavail);
> >>> +	avail_node_count = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
> >>> +
> >>> +	if (avail_node_count > user_block_count) {
> >>> +		buf->f_files = user_block_count;
> >>> +		buf->f_ffree = buf->f_bavail;
> >>
> >> f_ffree is limited both by remained free nid count and free block count, so it
> >> needs to change like this?
> > 
> > I thought both of them are same, since node block will consume user block. So,
> > we don't need to do min() again.
> 
> avail_node_count comes from total free nid counts which is limited with
> nid_bitmap size, buf->f_bavail comes from total user block count which
> can both cosumed by node and data. So the value of them may not be the same.

What I mean was, if avail_node_count is larger than user_block_count, we can
see buf->f_bavail is always smaller than avali_node_count - valid_node_count,
since node blocks concumes blocks as well.

> 
> Thanks,
> 
> > 
> >>
> >> if (avail_node_count > user_block_count)
> >> 	avail_node_count = user_block_count;
> >>
> >> buf->f_files = avail_node_count;
> >> buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
> >> 					buf->f_bavail);
> >>
> >> Thanks,
> >>
> >>> +	} else {
> >>> +		buf->f_files = avail_node_count;
> >>> +		buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
> >>> +					buf->f_bavail);
> >>> +	}
> >>>  
> >>>  	buf->f_namelen = F2FS_NAME_LEN;
> >>>  	buf->f_fsid.val[0] = (u32)id;
> >>>
> > 
> > ------------------------------------------------------------------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Linux-f2fs-devel mailing list
> > Linux-f2fs-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> > 

[toc] | [prev] | [next] | [standalone]

#1679282 — Re: [f2fs-dev] [PATCH 2/2] f2fs: report # of free inodes more precisely

On 2017/7/1 15:27, Jaegeuk Kim wrote:
> On 06/28, Chao Yu wrote:
>> On 2017/6/26 22:58, Jaegeuk Kim wrote:
>>> On 06/26, Chao Yu wrote:
>>>> Hi Jaegeuk,
>>>>
>>>> On 2017/6/25 0:25, Jaegeuk Kim wrote:
>>>>> If the partition is small, we don't need to report total # of inodes including
>>>>> hidden free nodes.
>>>>>
>>>>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
>>>>> ---
>>>>>  fs/f2fs/super.c | 14 +++++++++++---
>>>>>  1 file changed, 11 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
>>>>> index 8e39b850bfc0..3da6fb276f8b 100644
>>>>> --- a/fs/f2fs/super.c
>>>>> +++ b/fs/f2fs/super.c
>>>>> @@ -680,6 +680,7 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
>>>>>  	struct f2fs_sb_info *sbi = F2FS_SB(sb);
>>>>>  	u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
>>>>>  	block_t total_count, user_block_count, start_count, ovp_count;
>>>>> +	u64 avail_node_count;
>>>>>  
>>>>>  	total_count = le64_to_cpu(sbi->raw_super->block_count);
>>>>>  	user_block_count = sbi->user_block_count;
>>>>> @@ -692,9 +693,16 @@ static int f2fs_statfs(struct dentry *dentry, struct kstatfs *buf)
>>>>>  	buf->f_bfree = user_block_count - valid_user_blocks(sbi) + ovp_count;
>>>>>  	buf->f_bavail = user_block_count - valid_user_blocks(sbi);
>>>>>  
>>>>> -	buf->f_files = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
>>>>> -	buf->f_ffree = min(buf->f_files - valid_node_count(sbi),
>>>>> -							buf->f_bavail);
>>>>> +	avail_node_count = sbi->total_node_count - F2FS_RESERVED_NODE_NUM;
>>>>> +
>>>>> +	if (avail_node_count > user_block_count) {
>>>>> +		buf->f_files = user_block_count;
>>>>> +		buf->f_ffree = buf->f_bavail;
>>>>
>>>> f_ffree is limited both by remained free nid count and free block count, so it
>>>> needs to change like this?
>>>
>>> I thought both of them are same, since node block will consume user block. So,
>>> we don't need to do min() again.
>>
>> avail_node_count comes from total free nid counts which is limited with
>> nid_bitmap size, buf->f_bavail comes from total user block count which
>> can both cosumed by node and data. So the value of them may not be the same.
> 
> What I mean was, if avail_node_count is larger than user_block_count, we can
> see buf->f_bavail is always smaller than avali_node_count - valid_node_count,
> since node blocks concumes blocks as well.

Got you. :)

Reviewed-by: Chao Yu <yuchao0@huawei.com>

Thanks,

> 
>>
>> Thanks,
>>
>>>
>>>>
>>>> if (avail_node_count > user_block_count)
>>>> 	avail_node_count = user_block_count;
>>>>
>>>> buf->f_files = avail_node_count;
>>>> buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
>>>> 					buf->f_bavail);
>>>>
>>>> Thanks,
>>>>
>>>>> +	} else {
>>>>> +		buf->f_files = avail_node_count;
>>>>> +		buf->f_ffree = min(avail_node_count - valid_node_count(sbi),
>>>>> +					buf->f_bavail);
>>>>> +	}
>>>>>  
>>>>>  	buf->f_namelen = F2FS_NAME_LEN;
>>>>>  	buf->f_fsid.val[0] = (u32)id;
>>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Linux-f2fs-devel mailing list
>>> Linux-f2fs-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>>

[toc] | [prev] | [next] | [standalone]

#1674518 — Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

Hi Jaegeuk,

On 2017/6/25 0:25, Jaegeuk Kim wrote:
> - punch_hole
>  - fill_zero
>   - f2fs_lock_op
>   - get_new_data_page
>    - lock_page
> 
> - f2fs_write_data_pages
>  - lock_page
>  - do_write_data_page
>   - f2fs_lock_op

Good catch!

With this implementation, page writeback can fail due to concurrent checkpoint,
this will make fsync/atomic_commit which trigger synchronous write failed randomly.

How about unifying the lock order in punch_hole as one in writepages for regular
inode? We can add one more parameter in get_new_data_page to indicate whether
callee needs to lock cp_rwsem.

Thanks,

> 
> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> ---
>  fs/f2fs/data.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> index 7d3af48d34a9..9141bd19a902 100644
> --- a/fs/f2fs/data.c
> +++ b/fs/f2fs/data.c
> @@ -1404,8 +1404,9 @@ int do_write_data_page(struct f2fs_io_info *fio)
>  		}
>  	}
>  
> -	if (fio->need_lock == LOCK_REQ)
> -		f2fs_lock_op(fio->sbi);
> +	/* Deadlock due to between page->lock and f2fs_lock_op */
> +	if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
> +		return -EAGAIN;
>  
>  	err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
>  	if (err)
> 

[toc] | [prev] | [next] | [standalone]

#1674847 — Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

Hi Chao,

On 06/26, Chao Yu wrote:
> Hi Jaegeuk,
> 
> On 2017/6/25 0:25, Jaegeuk Kim wrote:
> > - punch_hole
> >  - fill_zero
> >   - f2fs_lock_op
> >   - get_new_data_page
> >    - lock_page
> > 
> > - f2fs_write_data_pages
> >  - lock_page
> >  - do_write_data_page
> >   - f2fs_lock_op
> 
> Good catch!
> 
> With this implementation, page writeback can fail due to concurrent checkpoint,
> this will make fsync/atomic_commit which trigger synchronous write failed randomly.
> 
> How about unifying the lock order in punch_hole as one in writepages for regular
> inode? We can add one more parameter in get_new_data_page to indicate whether
> callee needs to lock cp_rwsem.

Currently, there would be some places to keep cp_rwsem -> page.lock, which seems
not simple to change the lock order with page.lock -> cp_rwsem. IMO, we can retry
flushing data in f2fs_sync_file, once it gets -EAGAIN.

Any thoughts?

> 
> Thanks,
> 
> > 
> > Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> > ---
> >  fs/f2fs/data.c | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> > index 7d3af48d34a9..9141bd19a902 100644
> > --- a/fs/f2fs/data.c
> > +++ b/fs/f2fs/data.c
> > @@ -1404,8 +1404,9 @@ int do_write_data_page(struct f2fs_io_info *fio)
> >  		}
> >  	}
> >  
> > -	if (fio->need_lock == LOCK_REQ)
> > -		f2fs_lock_op(fio->sbi);
> > +	/* Deadlock due to between page->lock and f2fs_lock_op */
> > +	if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
> > +		return -EAGAIN;
> >  
> >  	err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
> >  	if (err)
> > 

[toc] | [prev] | [next] | [standalone]

#1674888 — Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

Hi Jaegeuk,

On 2017/6/26 22:54, Jaegeuk Kim wrote:
> Hi Chao,
> 
> On 06/26, Chao Yu wrote:
>> Hi Jaegeuk,
>>
>> On 2017/6/25 0:25, Jaegeuk Kim wrote:
>>> - punch_hole
>>>  - fill_zero
>>>   - f2fs_lock_op
>>>   - get_new_data_page
>>>    - lock_page
>>>
>>> - f2fs_write_data_pages
>>>  - lock_page
>>>  - do_write_data_page
>>>   - f2fs_lock_op
>>
>> Good catch!
>>
>> With this implementation, page writeback can fail due to concurrent checkpoint,
>> this will make fsync/atomic_commit which trigger synchronous write failed randomly.
>>
>> How about unifying the lock order in punch_hole as one in writepages for regular
>> inode? We can add one more parameter in get_new_data_page to indicate whether
>> callee needs to lock cp_rwsem.
> 
> Currently, there would be some places to keep cp_rwsem -> page.lock, which seems
> not simple to change the lock order with page.lock -> cp_rwsem. IMO, we can retry
> flushing data in f2fs_sync_file, once it gets -EAGAIN.
> 
> Any thoughts?

What about adding inode_lock in f2fs_sync_file to exclude other
foreground operation which have reversed lock order? Atomic_commit is OK
since it has inode_lock in its path.

Thanks,

> 
>>
>> Thanks,
>>
>>>
>>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
>>> ---
>>>  fs/f2fs/data.c | 5 +++--
>>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
>>> index 7d3af48d34a9..9141bd19a902 100644
>>> --- a/fs/f2fs/data.c
>>> +++ b/fs/f2fs/data.c
>>> @@ -1404,8 +1404,9 @@ int do_write_data_page(struct f2fs_io_info *fio)
>>>  		}
>>>  	}
>>>  
>>> -	if (fio->need_lock == LOCK_REQ)
>>> -		f2fs_lock_op(fio->sbi);
>>> +	/* Deadlock due to between page->lock and f2fs_lock_op */
>>> +	if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
>>> +		return -EAGAIN;
>>>  
>>>  	err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
>>>  	if (err)
>>>
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> 

[toc] | [prev] | [next] | [standalone]

#1679276 — Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

On 06/26, Chao Yu wrote:
> Hi Jaegeuk,
> 
> On 2017/6/26 22:54, Jaegeuk Kim wrote:
> > Hi Chao,
> > 
> > On 06/26, Chao Yu wrote:
> >> Hi Jaegeuk,
> >>
> >> On 2017/6/25 0:25, Jaegeuk Kim wrote:
> >>> - punch_hole
> >>>  - fill_zero
> >>>   - f2fs_lock_op
> >>>   - get_new_data_page
> >>>    - lock_page
> >>>
> >>> - f2fs_write_data_pages
> >>>  - lock_page
> >>>  - do_write_data_page
> >>>   - f2fs_lock_op
> >>
> >> Good catch!
> >>
> >> With this implementation, page writeback can fail due to concurrent checkpoint,
> >> this will make fsync/atomic_commit which trigger synchronous write failed randomly.
> >>
> >> How about unifying the lock order in punch_hole as one in writepages for regular
> >> inode? We can add one more parameter in get_new_data_page to indicate whether
> >> callee needs to lock cp_rwsem.
> > 
> > Currently, there would be some places to keep cp_rwsem -> page.lock, which seems
> > not simple to change the lock order with page.lock -> cp_rwsem. IMO, we can retry
> > flushing data in f2fs_sync_file, once it gets -EAGAIN.
> > 
> > Any thoughts?
> 
> What about adding inode_lock in f2fs_sync_file to exclude other
> foreground operation which have reversed lock order? Atomic_commit is OK
> since it has inode_lock in its path.

I have concerned about performance regression, if we do that.

> 
> Thanks,
> 
> > 
> >>
> >> Thanks,
> >>
> >>>
> >>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> >>> ---
> >>>  fs/f2fs/data.c | 5 +++--
> >>>  1 file changed, 3 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> >>> index 7d3af48d34a9..9141bd19a902 100644
> >>> --- a/fs/f2fs/data.c
> >>> +++ b/fs/f2fs/data.c
> >>> @@ -1404,8 +1404,9 @@ int do_write_data_page(struct f2fs_io_info *fio)
> >>>  		}
> >>>  	}
> >>>  
> >>> -	if (fio->need_lock == LOCK_REQ)
> >>> -		f2fs_lock_op(fio->sbi);
> >>> +	/* Deadlock due to between page->lock and f2fs_lock_op */
> >>> +	if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
> >>> +		return -EAGAIN;
> >>>  
> >>>  	err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
> >>>  	if (err)
> >>>
> > 
> > ------------------------------------------------------------------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Linux-f2fs-devel mailing list
> > Linux-f2fs-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> > 

[toc] | [prev] | [next] | [standalone]

#1679286 — Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

On 2017/7/1 15:28, Jaegeuk Kim wrote:
> On 06/26, Chao Yu wrote:
>> Hi Jaegeuk,
>>
>> On 2017/6/26 22:54, Jaegeuk Kim wrote:
>>> Hi Chao,
>>>
>>> On 06/26, Chao Yu wrote:
>>>> Hi Jaegeuk,
>>>>
>>>> On 2017/6/25 0:25, Jaegeuk Kim wrote:
>>>>> - punch_hole
>>>>>  - fill_zero
>>>>>   - f2fs_lock_op
>>>>>   - get_new_data_page
>>>>>    - lock_page
>>>>>
>>>>> - f2fs_write_data_pages
>>>>>  - lock_page
>>>>>  - do_write_data_page
>>>>>   - f2fs_lock_op
>>>>
>>>> Good catch!
>>>>
>>>> With this implementation, page writeback can fail due to concurrent checkpoint,
>>>> this will make fsync/atomic_commit which trigger synchronous write failed randomly.
>>>>
>>>> How about unifying the lock order in punch_hole as one in writepages for regular
>>>> inode? We can add one more parameter in get_new_data_page to indicate whether
>>>> callee needs to lock cp_rwsem.
>>>
>>> Currently, there would be some places to keep cp_rwsem -> page.lock, which seems
>>> not simple to change the lock order with page.lock -> cp_rwsem. IMO, we can retry
>>> flushing data in f2fs_sync_file, once it gets -EAGAIN.
>>>
>>> Any thoughts?
>>
>> What about adding inode_lock in f2fs_sync_file to exclude other
>> foreground operation which have reversed lock order? Atomic_commit is OK
>> since it has inode_lock in its path.
> 
> I have concerned about performance regression, if we do that.

I think fsync vs write or fsync vs fsync scenarios are unusual, so is
there any usecase?

Thanks,

> 
>>
>> Thanks,
>>
>>>
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
>>>>> ---
>>>>>  fs/f2fs/data.c | 5 +++--
>>>>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
>>>>> index 7d3af48d34a9..9141bd19a902 100644
>>>>> --- a/fs/f2fs/data.c
>>>>> +++ b/fs/f2fs/data.c
>>>>> @@ -1404,8 +1404,9 @@ int do_write_data_page(struct f2fs_io_info *fio)
>>>>>  		}
>>>>>  	}
>>>>>  
>>>>> -	if (fio->need_lock == LOCK_REQ)
>>>>> -		f2fs_lock_op(fio->sbi);
>>>>> +	/* Deadlock due to between page->lock and f2fs_lock_op */
>>>>> +	if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
>>>>> +		return -EAGAIN;
>>>>>  
>>>>>  	err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
>>>>>  	if (err)
>>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Linux-f2fs-devel mailing list
>>> Linux-f2fs-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>>

[toc] | [prev] | [next] | [standalone]

#1679368 — Re: [f2fs-dev] [PATCH 1/2] f2fs: avoid deadlock caused by lock order of page and lock_op

On 07/01, Chao Yu wrote:
> On 2017/7/1 15:28, Jaegeuk Kim wrote:
> > On 06/26, Chao Yu wrote:
> >> Hi Jaegeuk,
> >>
> >> On 2017/6/26 22:54, Jaegeuk Kim wrote:
> >>> Hi Chao,
> >>>
> >>> On 06/26, Chao Yu wrote:
> >>>> Hi Jaegeuk,
> >>>>
> >>>> On 2017/6/25 0:25, Jaegeuk Kim wrote:
> >>>>> - punch_hole
> >>>>>  - fill_zero
> >>>>>   - f2fs_lock_op
> >>>>>   - get_new_data_page
> >>>>>    - lock_page
> >>>>>
> >>>>> - f2fs_write_data_pages
> >>>>>  - lock_page
> >>>>>  - do_write_data_page
> >>>>>   - f2fs_lock_op
> >>>>
> >>>> Good catch!
> >>>>
> >>>> With this implementation, page writeback can fail due to concurrent checkpoint,
> >>>> this will make fsync/atomic_commit which trigger synchronous write failed randomly.
> >>>>
> >>>> How about unifying the lock order in punch_hole as one in writepages for regular
> >>>> inode? We can add one more parameter in get_new_data_page to indicate whether
> >>>> callee needs to lock cp_rwsem.
> >>>
> >>> Currently, there would be some places to keep cp_rwsem -> page.lock, which seems
> >>> not simple to change the lock order with page.lock -> cp_rwsem. IMO, we can retry
> >>> flushing data in f2fs_sync_file, once it gets -EAGAIN.
> >>>
> >>> Any thoughts?
> >>
> >> What about adding inode_lock in f2fs_sync_file to exclude other
> >> foreground operation which have reversed lock order? Atomic_commit is OK
> >> since it has inode_lock in its path.
> > 
> > I have concerned about performance regression, if we do that.
> 
> I think fsync vs write or fsync vs fsync scenarios are unusual, so is
> there any usecase?

Well, that'd be common to call multiple fsync calls at the same time.
Like dbench or tiotest?

> 
> Thanks,
> 
> > 
> >>
> >> Thanks,
> >>
> >>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>>>
> >>>>> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
> >>>>> ---
> >>>>>  fs/f2fs/data.c | 5 +++--
> >>>>>  1 file changed, 3 insertions(+), 2 deletions(-)
> >>>>>
> >>>>> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> >>>>> index 7d3af48d34a9..9141bd19a902 100644
> >>>>> --- a/fs/f2fs/data.c
> >>>>> +++ b/fs/f2fs/data.c
> >>>>> @@ -1404,8 +1404,9 @@ int do_write_data_page(struct f2fs_io_info *fio)
> >>>>>  		}
> >>>>>  	}
> >>>>>  
> >>>>> -	if (fio->need_lock == LOCK_REQ)
> >>>>> -		f2fs_lock_op(fio->sbi);
> >>>>> +	/* Deadlock due to between page->lock and f2fs_lock_op */
> >>>>> +	if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
> >>>>> +		return -EAGAIN;
> >>>>>  
> >>>>>  	err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
> >>>>>  	if (err)
> >>>>>
> >>>
> >>> ------------------------------------------------------------------------------
> >>> Check out the vibrant tech community on one of the world's most
> >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >>> _______________________________________________
> >>> Linux-f2fs-devel mailing list
> >>> Linux-f2fs-devel@lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> >>>

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web