Discussion:
How to stop systemd-udevd reading a device after dd
(too old to reply)
Akira Hayakawa
2017-10-12 16:01:01 UTC
Permalink
Raw Message
I have a device /dev/sdb1 and let's trace the block request by blktrace

$ sudo blktrace -d /dev/sdb1

When I write 4KB using dd
$ sudo dd if=/dev/zero of=/dev/sdb1 oflag=direct bs=4k count=1

The block trace (after blkparsed) is write request as expected
8,17 2 2 0.000003171 5930 Q WS 2048 + 8 [dd]

followed by a unexpected read from systemd-udevd
8,17 7 2 0.001755563 5931 Q R 2048 + 8 [systemd-udevd]

My first question is what is: this read request?

And I want to stop the read request because it makes it difficult to test kernel code.
So the second question is: how can I stop the read request?
Actually I found this behavior when I test my module https://github.com/akiradeveloper/dm-writeboost
Since I need to create/remove devices during the tests, there is no choice to stop udev.service.

FYI, I am using Debian 9.2

Thanks,
- Akira
Mantas Mikulėnas
2017-10-13 05:06:10 UTC
Permalink
Raw Message
Post by Akira Hayakawa
I have a device /dev/sdb1 and let's trace the block request by blktrace
$ sudo blktrace -d /dev/sdb1
When I write 4KB using dd
$ sudo dd if=/dev/zero of=/dev/sdb1 oflag=direct bs=4k count=1
The block trace (after blkparsed) is write request as expected
8,17 2 2 0.000003171 5930 Q WS 2048 + 8 [dd]
followed by a unexpected read from systemd-udevd
8,17 7 2 0.001755563 5931 Q R 2048 + 8 [systemd-udevd]
My first question is what is: this read request?
When a device node is closed for writing, the kernel generates a 'change'
uevent for it. This prompts udev to re-read filesystems and update its
cache of labels, UUIDs, and other information. Tools like `lsblk` or
/dev/disk/by-{id,uuid,label}/* use the udev database.

And I want to stop the read request because it makes it difficult to test
Post by Akira Hayakawa
kernel code.
So the second question is: how can I stop the read request?
gparted seems to achieve this by masking all .rules files it can find (by
creating 0-byte versions under /run/udev/rules.d).

Not sure what else to recommend. I thought there used to be inhibit mode
under `udevadm control`, but I can't find it anymore.
Post by Akira Hayakawa
Actually I found this behavior when I test my module https://github.com/
akiradeveloper/dm-writeboost
Since I need to create/remove devices during the tests, there is no choice
to stop udev.service.
The kernel itself adds/removes device nodes under /dev. Udev only manages
auxiliary stuff like symlinks (like /dev/disk/by-*), permissions, and such.
So you *could* survive without udev in case of emergency.
--
Mantas Mikulėnas <***@gmail.com>
Lennart Poettering
2017-10-24 08:33:49 UTC
Permalink
Raw Message
Post by Mantas Mikulėnas
gparted seems to achieve this by masking all .rules files it can find (by
creating 0-byte versions under /run/udev/rules.d).
Urks.

Somebody should tell them about BSD file locks.

Lennart
--
Lennart Poettering, Red Hat
Mantas Mikulėnas
2017-10-26 12:31:01 UTC
Permalink
Raw Message
Post by Lennart Poettering
Post by Mantas Mikulėnas
gparted seems to achieve this by masking all .rules files it can find (by
creating 0-byte versions under /run/udev/rules.d).
Urks.
Somebody should tell them about BSD file locks.
That might not be enough for their purposes, as it also tries to pause
udisks2, udisks, devicekit-disks, and even hal.

https://github.com/GNOME/gparted/blob/master/gparted.in#L168
--
Mantas Mikulėnas <***@gmail.com>
Lennart Poettering
2017-10-26 12:49:18 UTC
Permalink
Raw Message
Post by Mantas Mikulėnas
Post by Lennart Poettering
Post by Mantas Mikulėnas
gparted seems to achieve this by masking all .rules files it can find (by
creating 0-byte versions under /run/udev/rules.d).
Urks.
Somebody should tell them about BSD file locks.
That might not be enough for their purposes, as it also tries to pause
udisks2, udisks, devicekit-disks, and even hal.
All of which listen to udev afaik, and given that udev honours the BSD
locks, we should be good hence.

Also, all of those are obsolete these days, so it might be time to update.

Lennart
--
Lennart Poettering, Red Hat
Michal Sekletar
2017-10-13 09:01:55 UTC
Permalink
Raw Message
Post by Akira Hayakawa
And I want to stop the read request because it makes it difficult to test kernel code.
So the second question is: how can I stop the read request?
You can install local udev rule that will disable udev's monitoring of
the block device.

# cat /etc/udev/rules.d/61-no-watch-sdb.rules
ACTION!="remove", SUBSYSTEM=="block", KERNEL=="sdb*", OPTIONS:="nowatch"
# udevadm control --reload-rules

Udev watches exist because block layer doesn't have any generic
mechanism that could be used by the tools (e2fsprogs and friends) to
make sure that udev picks up changes that they did, e.g. change in
filesystem label.

Michal
Akira Hayakawa
2017-10-13 11:08:27 UTC
Permalink
Raw Message
Hi,

Thanks.

The udev rule worked. But only for the unexpected read after write
(or close the device by dd maybe)
Because the device I want to remove udev control is device-mapper devices
which have prefix dm-, I changed the KERNEL in the line from sdb* to dm* then worked.

But still I have problem: There are still reads after dmsetup create
and dmsetup resume. I suppose these operations send uevents to udev
but I guess the nowatch line in udev rules should suppress them.

What do you think the remaining problems are?

- Akira
Post by Michal Sekletar
Post by Akira Hayakawa
And I want to stop the read request because it makes it difficult to test kernel code.
So the second question is: how can I stop the read request?
You can install local udev rule that will disable udev's monitoring of
the block device.
# cat /etc/udev/rules.d/61-no-watch-sdb.rules
ACTION!="remove", SUBSYSTEM=="block", KERNEL=="sdb*", OPTIONS:="nowatch"
# udevadm control --reload-rules
Udev watches exist because block layer doesn't have any generic
mechanism that could be used by the tools (e2fsprogs and friends) to
make sure that udev picks up changes that they did, e.g. change in
filesystem label.
Michal
Akira Hayakawa
2017-10-14 02:04:58 UTC
Permalink
Raw Message
I've investigate myself what uevent are sent after each command.
I use "udevadm monitor" to watch the event sent from kernel.

The experiment starts from removing the existing device but
if it's placed at last, the uevents are the same.

What's very curious to me is that the ACTION!="remove" in my rule
only ignores the uevent from dd but the rest effects.
The uevent after dmsetup resume is considered even though the
uevent shapes the same as dd.

All I want to do is suppress the read after these commands.
I think it's health check by reading randomly chosen blocks.
I've observed that more than 100 of 4KB reads are submitted to my
logical device after creation and that hinders my tests as they
expect these extra IOs aren't existing. (more precisely saying
it's not black box test but white box that uses the internal stats
in assertions)

# dmsetup remove
KERNEL[4202.177671] remove /devices/virtual/block/dm-0 (block)
KERNEL[4202.205822] remove /kernel/slab/dmwb_buf_8 (slab)
KERNEL[4202.206786] remove /kernel/slab/bio-3 (slab)
KERNEL[4202.250279] remove /devices/virtual/bdi/251:0 (bdi)
KERNEL[4202.250776] remove /devices/virtual/block/dm-0 (block)

# dmsetup create
KERNEL[4226.692607] add /devices/virtual/bdi/251:0 (bdi)
KERNEL[4226.692852] add /devices/virtual/block/dm-0 (block)
KERNEL[4226.695092] add /kernel/slab/dmwb_buf_8 (slab)
KERNEL[4229.107766] add /kernel/slab/bio-3 (slab)

# dd (write)
KERNEL[4229.108196] change /devices/virtual/block/dm-0 (block)

# dmsetup suspend
# dmsetup resume
KERNEL[4271.409068] change /devices/virtual/block/dm-0 (block)

- Akira
Lennart Poettering
2017-10-24 08:33:15 UTC
Permalink
Raw Message
Post by Akira Hayakawa
I have a device /dev/sdb1 and let's trace the block request by blktrace
$ sudo blktrace -d /dev/sdb1
When I write 4KB using dd
$ sudo dd if=/dev/zero of=/dev/sdb1 oflag=direct bs=4k count=1
The block trace (after blkparsed) is write request as expected
8,17 2 2 0.000003171 5930 Q WS 2048 + 8 [dd]
followed by a unexpected read from systemd-udevd
8,17 7 2 0.001755563 5931 Q R 2048 + 8 [systemd-udevd]
My first question is what is: this read request?
And I want to stop the read request because it makes it difficult to test kernel code.
So the second question is: how can I stop the read request?
If you want exclusive access to a block device and don't want udev to
step in, then simply take a BSD file lock on it (i.e. flock(2)), and
udev won't probe it.

This is not particularly well documented (one could say: not at all),
but it's the official way really, and very UNIXish I'd claim.

Lennart
--
Lennart Poettering, Red Hat
Andrei Borzenkov
2017-11-13 04:02:09 UTC
Permalink
Raw Message
Post by Lennart Poettering
Post by Akira Hayakawa
I have a device /dev/sdb1 and let's trace the block request by blktrace
$ sudo blktrace -d /dev/sdb1
When I write 4KB using dd
$ sudo dd if=/dev/zero of=/dev/sdb1 oflag=direct bs=4k count=1
The block trace (after blkparsed) is write request as expected
8,17 2 2 0.000003171 5930 Q WS 2048 + 8 [dd]
followed by a unexpected read from systemd-udevd
8,17 7 2 0.001755563 5931 Q R 2048 + 8 [systemd-udevd]
My first question is what is: this read request?
And I want to stop the read request because it makes it difficult to test kernel code.
So the second question is: how can I stop the read request?
If you want exclusive access to a block device and don't want udev to
step in, then simply take a BSD file lock on it (i.e. flock(2)), and
udev won't probe it.
How do you take exclusive lock on device that does not even exist yet
(like in case of dmsetup create)?

Loading...