Page MenuHomeFreeBSD

fsu (Fedor Uporov)
User

Projects

User does not belong to any projects.

User Details

User Since
Mar 27 2017, 5:46 PM (133 w, 5 d)

Recent Activity

Jul 27 2019

fsu committed rS350385: MFC r349800,r349801,r349802:.
MFC r349800,r349801,r349802:
Jul 27 2019, 7:29 PM
fsu committed rS350384: MFC r349800,r349801:.
MFC r349800,r349801:
Jul 27 2019, 7:29 PM

Jul 7 2019

fsu committed rS349802: Add additional check for 'blocks per group' and 'fragments per group'….
Add additional check for 'blocks per group' and 'fragments per group'…
Jul 7 2019, 8:58 AM
fsu committed rS349801: Remove ufs fragments logic..
Remove ufs fragments logic.
Jul 7 2019, 8:56 AM
fsu committed rS349800: Remove unneeded mount point unlock call..
Remove unneeded mount point unlock call.
Jul 7 2019, 8:54 AM

Jun 10 2019

fsu accepted D20571: Add a sentence about full ext4 read/write support the filesystems chapter in the handbook.

I am not sure about "filesystem has full read and write support for ext4" because there some ext4 features, that uses rarely and we does not support it.
But may be do not dig into the details in this place.

Jun 10 2019, 2:25 PM

Apr 30 2019

fsu committed rS346955: MFC r346267, 346269:.
MFC r346267, 346269:
Apr 30 2019, 9:10 AM

Apr 27 2019

fsu abandoned D13487: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_*..

Indeed, this seems to have been abandoned and recreated as D13528, then committed as r338152. @fsu is there any reason not to close this review?

Apr 27 2019, 4:17 PM

Apr 16 2019

fsu committed rS346269: ext2fs: Initial version of DTrace support..
ext2fs: Initial version of DTrace support.
Apr 16 2019, 11:37 AM
fsu committed rS346267: ext2fs: Initial version of DTrace support..
ext2fs: Initial version of DTrace support.
Apr 16 2019, 11:20 AM
fsu closed D19848: ext2fs: Dtrace support..
Apr 16 2019, 11:20 AM

Apr 12 2019

fsu added a comment to D19848: ext2fs: Dtrace support..
In D19848#426959, @gnn wrote:

Some general comments.

  1. Can you move all the DEFINE statements to the top of the file rather than interspresing them throughout? I know you're defining them near use but the general way we have coded these is to put the DECLARES and DEFINES at the top of the files.

Ok, I will add this change before landing.

  1. The name "trace" is going to be a bit confusing to users as they already think they're tracing the system. You could drop the "trace" altogether and see if that makes sense.

I prefer to leave it as is to keep consistency with fusefs dtrace probes naming. See: https://reviews.freebsd.org/D19667

Otherwise this looks like a great addition, and thank you for doing it.

Apr 12 2019, 7:03 AM

Apr 10 2019

fsu added a comment to D19848: ext2fs: Dtrace support..
In D19848#426730, @pfg wrote:

I still hope to hear from the DTrace experts, but I like this very much. Thanks!

Apr 10 2019, 6:12 PM

Apr 9 2019

fsu added inline comments to D19848: ext2fs: Dtrace support..
Apr 9 2019, 7:39 AM

Apr 8 2019

fsu retitled D19848: ext2fs: Dtrace support. from ext2fs: Drace support. to ext2fs: Dtrace support..
Apr 8 2019, 5:20 PM
fsu created D19848: ext2fs: Dtrace support..
Apr 8 2019, 5:19 PM

Mar 29 2019

fsu added a comment to D19599: Add a UFS-specific implementation for lseek(SEEK_DATA)..
In D19599#423359, @kib wrote:
In D19599#423338, @fsu wrote:
In D19599#423330, @fsu wrote:

Hi, Mark.
Thanks a lot for adding me to review.
But could you please provide some information, how it is possible to test FIOSEEKDATA/FIOSEEKHOLE from user-space side.
I mean, did you write some user-space code to test it or some unit-tests exist somewhere?

I did not modify SEEK_HOLE. I tested SEEK_DATA with a program that truncates an empty file to a random length, and then calls SEEK_DATA on the file with a random starting offset, and verifies that the value returned is the starting offset of the last LBN in the file, since UFS seems to always put a data block at the end of a sparse file, even if the inode size % block size is 0. I also ran a private instance of syzkaller on the patch and verified that it exercises most of the branches in the new function (currently it cannot trigger errors from readindir(), for example).
I started writing some tests, but discarded them. It is hard to write useful tests since the implementation of file holes is highly filesystem-dependent, and even for UFS there is no blessed procedure for creating holes in a file AFAIK.

What you described above *is* the correct way to create holes on UFS. Seek after the EOF and write single byte, then all the content between old EOF and new block containing that byte is hole. The only detail is that if the byte is in direct block, then you might get fragment at the end, instead of the full block (does not matter for the tests).

So any tests for this will be coupled to filesystem internals.

Also, it is interesting how faster it became.

I did not attempt to compare them. The old algorithm is O(size of file) and the new one is effectively constant-time as long as the inode does not reference indirect blocks that are full of zeroes. With a sufficiently large sparse file the old algorithm will not complete in any reasonable amount of time.

I expected that some user-space tests exist, which could be possible to apply to ext2fs to ensure, that the implementation is compatible with ufs.

https://gist.github.com/5319d0d7d52e0f9199ddf2b5a75a6438
https://gist.github.com/a4f9c80a7699151e7f2e5ba051740aa7
This is what I used for vn_bmap_seekhole() implementation.

Ok, thank you. This is, what I tried to find, to avoid implement it from scratch.

But ok, I will try to find something and check it on ufs too.
Thanks.

Mar 29 2019, 9:15 PM
fsu added a comment to D19599: Add a UFS-specific implementation for lseek(SEEK_DATA)..
In D19599#423330, @fsu wrote:

Hi, Mark.
Thanks a lot for adding me to review.
But could you please provide some information, how it is possible to test FIOSEEKDATA/FIOSEEKHOLE from user-space side.
I mean, did you write some user-space code to test it or some unit-tests exist somewhere?

I did not modify SEEK_HOLE. I tested SEEK_DATA with a program that truncates an empty file to a random length, and then calls SEEK_DATA on the file with a random starting offset, and verifies that the value returned is the starting offset of the last LBN in the file, since UFS seems to always put a data block at the end of a sparse file, even if the inode size % block size is 0. I also ran a private instance of syzkaller on the patch and verified that it exercises most of the branches in the new function (currently it cannot trigger errors from readindir(), for example).
I started writing some tests, but discarded them. It is hard to write useful tests since the implementation of file holes is highly filesystem-dependent, and even for UFS there is no blessed procedure for creating holes in a file AFAIK. So any tests for this will be coupled to filesystem internals.

Also, it is interesting how faster it became.

I did not attempt to compare them. The old algorithm is O(size of file) and the new one is effectively constant-time as long as the inode does not reference indirect blocks that are full of zeroes. With a sufficiently large sparse file the old algorithm will not complete in any reasonable amount of time.

Mar 29 2019, 3:49 PM
fsu added a comment to D19599: Add a UFS-specific implementation for lseek(SEEK_DATA)..

Thanks a lot for adding me to review.
But could you please provide some information, how it is possible to test FIOSEEKDATA/FIOSEEKHOLE from user-space side.
I mean, did you write some user-space code to test it or some unit-tests exist somewhere?

Mar 29 2019, 3:05 PM

Mar 18 2019

fsu committed rS345272: MFC: r344757:.
MFC: r344757:
Mar 18 2019, 12:34 PM
fsu committed rS345271: MFC: r344752:.
MFC: r344752:
Mar 18 2019, 12:31 PM
fsu committed rS345270: MFC: r344754:.
MFC: r344754:
Mar 18 2019, 12:26 PM
fsu committed rS345268: MFC: r344756, r345179:.
MFC: r344756, r345179:
Mar 18 2019, 12:22 PM
fsu committed rS345267: MFC: r344755:.
MFC: r344755:
Mar 18 2019, 12:16 PM
fsu committed rS345266: MFC: r344753:.
MFC: r344753:
Mar 18 2019, 12:09 PM
fsu committed rS345265: MFC r344751:.
MFC r344751:
Mar 18 2019, 12:04 PM

Mar 15 2019

fsu committed rS345179: Remove unneeded mount point unlock function calls..
Remove unneeded mount point unlock function calls.
Mar 15 2019, 11:49 AM

Mar 4 2019

fsu committed rS344757: Fix double free in case of mount error..
Fix double free in case of mount error.
Mar 4 2019, 11:34 AM
fsu closed D19385: Fix double free in case of mount error..
Mar 4 2019, 11:34 AM
fsu committed rS344756: Do not read the on-disk inode in case of vnode allocation..
Do not read the on-disk inode in case of vnode allocation.
Mar 4 2019, 11:27 AM
fsu closed D19327: Do not read the on-disk inode in case of vnode allocation..
Mar 4 2019, 11:27 AM
fsu committed rS344755: Fix integer overflow possibility..
Fix integer overflow possibility.
Mar 4 2019, 11:19 AM
fsu closed D19326: Fix integer overflow possibility..
Mar 4 2019, 11:19 AM
fsu committed rS344754: Do not panic if inode bitmap is corrupted..
Do not panic if inode bitmap is corrupted.
Mar 4 2019, 11:12 AM
fsu closed D19325: Do not panic if inode bitmap is corrupted..
Mar 4 2019, 11:12 AM
fsu committed rS344753: Validate block bitmaps..
Validate block bitmaps.
Mar 4 2019, 11:01 AM
fsu closed D19324: Validate block bitmaps..
Mar 4 2019, 11:01 AM
fsu closed D19323: Add additional on-disk inode checks..
Mar 4 2019, 10:55 AM
fsu committed rS344752: Add additional on-disk inode checks..
Add additional on-disk inode checks.
Mar 4 2019, 10:55 AM
fsu committed rS344751: Make superblock reading logic more strict..
Make superblock reading logic more strict.
Mar 4 2019, 10:42 AM
fsu closed D19322: Make superblock reading logic more strict..
Mar 4 2019, 10:42 AM

Feb 27 2019

fsu added a comment to D19327: Do not read the on-disk inode in case of vnode allocation..
In D19327#413791, @pfg wrote:

Note why we specifically set ip->i_flags = 0 since r294652:

Feb 27 2019, 4:32 PM
fsu added a comment to D19325: Do not panic if inode bitmap is corrupted..
In D19325#414951, @pfg wrote:

We should probably keep this panic for consistency with UFS, which also panics on this case.

Feb 27 2019, 4:32 PM
fsu created D19385: Fix double free in case of mount error..
Feb 27 2019, 11:37 AM

Feb 24 2019

fsu added a reviewer for D19326: Fix integer overflow possibility.: pfg.
Feb 24 2019, 1:16 PM
fsu created D19327: Do not read the on-disk inode in case of vnode allocation..
Feb 24 2019, 1:16 PM
fsu created D19326: Fix integer overflow possibility..
Feb 24 2019, 1:15 PM
fsu created D19325: Do not panic if inode bitmap is corrupted..
Feb 24 2019, 1:14 PM
fsu created D19324: Validate block bitmaps..
Feb 24 2019, 1:14 PM
fsu created D19323: Add additional on-disk inode checks..
Feb 24 2019, 1:14 PM
fsu created D19322: Make superblock reading logic more strict..
Feb 24 2019, 1:13 PM

Jan 13 2019

fsu committed rS342999: MFC r342933:.
MFC r342933:
Jan 13 2019, 12:13 PM
fsu committed rS342998: MFC r342933:.
MFC r342933:
Jan 13 2019, 12:13 PM

Jan 11 2019

fsu committed rS342933: Fix errno values returned from DUMMY_XATTR linuxulator calls.
Fix errno values returned from DUMMY_XATTR linuxulator calls
Jan 11 2019, 7:58 AM
fsu closed D18812: Fix errno values returned from DUMMY_XATTR linuxulator calls..
Jan 11 2019, 7:58 AM

Jan 10 2019

fsu added a comment to D18812: Fix errno values returned from DUMMY_XATTR linuxulator calls..

Thanks.
The duplication of these stubs is rather unfortunate...

Jan 10 2019, 5:46 PM
fsu created D18812: Fix errno values returned from DUMMY_XATTR linuxulator calls..
Jan 10 2019, 5:24 PM

Aug 21 2018

fsu committed rS338153: FUSE extattrs: fix issue when neither uio nor size were not passed to VOP_*….
FUSE extattrs: fix issue when neither uio nor size were not passed to VOP_*…
Aug 21 2018, 6:50 PM
fsu closed D13737: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_* (cosmetic only)..
Aug 21 2018, 6:50 PM
fsu committed rS338152: FUSE extattrs: fix issue when neither uio nor size were not passed to VOP_*..
FUSE extattrs: fix issue when neither uio nor size were not passed to VOP_*.
Aug 21 2018, 6:40 PM
fsu closed D13528: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_* (logic only)..
Aug 21 2018, 6:40 PM
fsu committed rS338151: Change unused inodes counters behavior in the cylinder groups..
Change unused inodes counters behavior in the cylinder groups.
Aug 21 2018, 6:39 PM
fsu committed rS338150: Fix directory blocks checksum updating logic..
Fix directory blocks checksum updating logic.
Aug 21 2018, 6:39 PM

Aug 9 2018

fsu added inline comments to D13528: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_* (logic only)..
Aug 9 2018, 8:06 AM

Aug 8 2018

fsu committed rS337454: Split the dir_index and dir_nlink features..
Split the dir_index and dir_nlink features.
Aug 8 2018, 12:08 PM
fsu committed rS337453: Fix directory blocks checksum updating logic..
Fix directory blocks checksum updating logic.
Aug 8 2018, 12:08 PM

May 13 2018

fsu committed rS333586: Fix directory blocks checksumming..
Fix directory blocks checksumming.
May 13 2018, 7:48 PM
fsu closed D15396: Fix directory blocks checksumming..
May 13 2018, 7:48 PM
fsu committed rS333585: Fix on-disk inode checksum calculation logic..
Fix on-disk inode checksum calculation logic.
May 13 2018, 7:29 PM
fsu closed D15395: Fix on-disk inode checksum calculation logic..
May 13 2018, 7:29 PM
fsu committed rS333584: Fix EXT2FS_DEBUG definition usage..
Fix EXT2FS_DEBUG definition usage.
May 13 2018, 7:19 PM
fsu closed D15394: Fix EXT2FS_DEBUG definition usage..
May 13 2018, 7:19 PM
fsu added inline comments to D15396: Fix directory blocks checksumming..
May 13 2018, 8:04 AM
fsu added inline comments to D15395: Fix on-disk inode checksum calculation logic..
May 13 2018, 8:02 AM
fsu added inline comments to D15394: Fix EXT2FS_DEBUG definition usage..
May 13 2018, 8:00 AM

May 11 2018

fsu created D15396: Fix directory blocks checksumming..
May 11 2018, 6:54 PM
fsu created D15395: Fix on-disk inode checksum calculation logic..
May 11 2018, 6:54 PM
fsu created D15394: Fix EXT2FS_DEBUG definition usage..
May 11 2018, 6:54 PM

Feb 5 2018

fsu accepted D14193: ext2fs: Tidy ups for variable assignment..

I am more prefer to initialize stack variables immediately after declaration, but, seems like it is not so perfect practice.
Ok.

Feb 5 2018, 7:53 AM

Jan 29 2018

fsu committed rS328566: Fix mistake in case of zeroed inode check..
Fix mistake in case of zeroed inode check.
Jan 29 2018, 10:15 PM
fsu committed rS328564: Add flex_bg/meta_bg features RW support..
Add flex_bg/meta_bg features RW support.
Jan 29 2018, 9:54 PM
fsu closed D13964: Add flex_bg/meta_bg features RW support. .
Jan 29 2018, 9:54 PM

Jan 23 2018

fsu accepted D14017: Remove some apparently unused variables..

Yep, accepted.
The code of extents part should be slightly cleaned up, it is my "nearest" future plans)
Also, it would be great to little bit rename variables inside the ext2_ei_csum(), but I will do it by myself later to avoid influence to logic.

Jan 23 2018, 7:56 AM

Jan 19 2018

fsu added a comment to D13964: Add flex_bg/meta_bg features RW support. .
In D13964#293380, @pfg wrote:
In D13964#293266, @fsu wrote:
In D13964#293256, @pfg wrote:

Hmm.. it seems like we are getting to a rather complete ext4 implementation, which is positive. On the downside the code is diverging from UFS. We just can't have both I guess.

There are another features like RO_COMPAT_QUOTA, INCOMPAT_INLINE_DATA, etc, but from my current point of view, it is not so, important, if it will be required to implement it. I think it a not a problem.
The only feature, which will be difficult to implement is RO_COMPAT_BIGALLOC, but I hope it will be not widely used by linux.

I haven't looked at ext4 for a while but most of those features seem to require some obscure collaboration from the kernel, that is out of the scope of the driver. There's also the issue that ost of the interesting enhancements in linux assume there is journalling.

I am not sure, that journal feature is required for non-native file system, also it will be difficult to test it.

In any case , let me mention that EXT4F_RO_INCOMPAT_SUPP was created as a dirty hack: to workaround for features that we were not planning to support at all but that we needed to fake in order to read the partition.
We should remove EXT4F_RO_INCOMPAT_SUPP as it has fulfilled its purpose.

Yep, I will prepare the commit to remove it.

Have you tested performance improvements from these features? They could be the basis for a conference paper :).

I am not sure, that any of implemented ext4 features will provide the performance improvements, because these features a not targeted for it, so performance measurement is not actual.

Hmm.. right. Most of the features are actually enhancements for fsck: checksumming will probably hit performance.

Also, if you mean, comparing it with linux driver, freebsd will fail in all cases) because, it is "native" file-system for linux and freebsd driver implementation is slightly simplified comparing with it.

The objectives have been different but at some point our ext2 had the same basic performance of UFS1 and linux's ext2, which was actually faster than ext3. We have never made comparisons with ext4.

The comparing performance of ufs and extfs is an interesting. I sure, that ufs will be faster, but how many times... May be will make some measurements little bit later.

Jan 19 2018, 6:27 PM
fsu added a comment to D13964: Add flex_bg/meta_bg features RW support. .
In D13964#293256, @pfg wrote:

Hmm.. it seems like we are getting to a rather complete ext4 implementation, which is positive. On the downside the code is diverging from UFS. We just can't have both I guess.

There are another features like RO_COMPAT_QUOTA, INCOMPAT_INLINE_DATA, etc, but from my current point of view, it is not so, important, if it will be required to implement it. I think it a not a problem.
The only feature, which will be difficult to implement is RO_COMPAT_BIGALLOC, but I hope it will be not widely used by linux.

In any case , let me mention that EXT4F_RO_INCOMPAT_SUPP was created as a dirty hack: to workaround for features that we were not planning to support at all but that we needed to fake in order to read the partition.
We should remove EXT4F_RO_INCOMPAT_SUPP as it has fulfilled its purpose.

Yep, I will prepare the commit to remove it.

Have you tested performance improvements from these features? They could be the basis for a conference paper :).

I am not sure, that any of implemented ext4 features will provide the performance improvements, because these features a not targeted for it, so performance measurement is not actual.
Also, if you mean, comparing it with linux driver, freebsd will fail in all cases) because, it is "native" file-system for linux and freebsd driver implementation is slightly simplified comparing with it.

Jan 19 2018, 7:24 AM

Jan 18 2018

fsu created D13964: Add flex_bg/meta_bg features RW support. .
Jan 18 2018, 2:08 PM

Jan 14 2018

fsu committed rS327977: Add metadata_csum feature support..
Add metadata_csum feature support.
Jan 14 2018, 8:46 PM
fsu closed D13810: Add metadata_csum feature support..
Jan 14 2018, 8:46 PM

Jan 10 2018

fsu added a comment to D13810: Add metadata_csum feature support..
In D13810#289909, @pfg wrote:
In D13810#289828, @fsu wrote:
In D13810#289543, @pfg wrote:

A thought : would it be possible to checksum only what we are going to write to disk?
Data will not, generally, corrupt in memory and the kernel has no use for the checksums at runtime so we only need to write stuff when it's going to be converted to the on-disk format in ext2_inode_cnv.c.

All fields, which are mentioned in SUMMERY, should be written to disk. If it will not be updated, we will have errors reported by e2fsck, or bunch of error messages from driver on linux side.

I am not complaining at all about what is being checksummed: I understand you want to checksum everything that is needed. The main doubt is when to checksum.
There are two possibilities: you either checksum everything as soon as you get it or you checksum at the last moment before writing to disk. Both options have their advantages It seems like the checksumming is happening all over the filesystem so you are doing the former, I was only wondering if we could do all the checksums in a single place, right when we are converting to the ext2fs format for writing.
I am not sure if such thing is poosible though, and your change is not incorrect so I'll let you think about it but I'll approve the patch.

Jan 10 2018, 4:16 PM
fsu added a comment to D13810: Add metadata_csum feature support..
In D13810#289543, @pfg wrote:

A thought : would it be possible to checksum only what we are going to write to disk?
Data will not, generally, corrupt in memory and the kernel has no use for the checksums at runtime so we only need to write stuff when it's going to be converted to the on-disk format in ext2_inode_cnv.c.

Jan 10 2018, 7:53 AM

Jan 9 2018

fsu created D13810: Add metadata_csum feature support..
Jan 9 2018, 10:25 AM
fsu added inline comments to D13528: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_* (logic only)..
Jan 9 2018, 6:50 AM
fsu updated the diff for D13528: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_* (logic only)..
  • Declare struct fuse_listxattr_* to not use fuse_getxattr_* in case of list().
  • Remove ENOSYS error check in case of second fdisp_wait_answ() call under list().
Jan 9 2018, 6:37 AM

Jan 8 2018

fsu added inline comments to D13528: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_* (logic only)..
Jan 8 2018, 9:15 AM
fsu updated the diff for D13528: fuse extattrs: fix issue when neither uio nor size were not passed to VOP_* (logic only)..

More deep reinitialization of struct fuse_dispatcher fdi before second call to fdisp_wait_answ() under fuse_vnop_listextattr().

Jan 8 2018, 8:57 AM

Jan 5 2018

fsu committed rS327585: MFC r326807:.
MFC r326807:
Jan 5 2018, 10:04 AM
fsu closed D11530: ext2fs: Add 64bit feature support.
Jan 5 2018, 10:04 AM
fsu committed rS327584: Add 64bit feature support..
Add 64bit feature support.
Jan 5 2018, 10:04 AM

Jan 3 2018

fsu updated the diff for D13267: Export extattr_*vp() functions to allow to use it in the linuxulator in future .

Ok, let's try to proceed...
Here is very initial version of vfs_extattr.c, where the io faults are handled.
It even somehow does not works with ufs, but works with ext2fs, I did not dig in this problem for now, because it is needed to figure out the direction and next steps for extattrs topic, the testing procedure will include all available filesystems with extattrs support and will be developed later.
So, @kib, could you please review it and provide the comments.

Jan 3 2018, 12:06 PM
fsu added inline comments to D11530: ext2fs: Add 64bit feature support.
Jan 3 2018, 7:44 AM