[systemd-devel] device units rely on udev rules?

Post by Scott James Remnant
Another question I wasn't able to find an answer to in the documentation
I've read so far.
The use of device units seems to very much rely on udevd running on the
system, and not only that, udev rules having been parsed for the device and
a systemd tag "set" in the udevdb. udev obviously starts after systemd, and
systemd starts after the kernel.

Correct.

Post by Scott James Remnant
This means there are a large number of devices already known to the kernel
at the point that systemd starts, especially if you build the drivers into
the kernel for those devices. It's possible to get going straight away with
those devices. But relying on udevd tagging them means you end up waiting
around for udevd to start, and worse! since udevd doesn't apply rules to
existing devices on startup, you have to wait around for "udevadm trigger"
to be run.

That's actually dead fast, and systemd picks up those devices as they
show up. The trigger can hence finish after we already preceeded
booting. In fact, with the new netlink actviation in newest udev and
systemd we can start the trigger at the same time as udev itself.

We cannot really mount file systems before their block devices have
shown up and have been probed, and we exactly wait for that, and no
longer. To do fsck/mount we need udev to have run for that device,
there's no realistic way around that.

Post by Scott James Remnant
Given that devtmpfs means the device nodes are already available, and
systemd will be merging the ACL handling from CK/udev into itself, why does
systemd rely on udev coldplugging in this way? All that's left for udevd to
do at this point is "device groups" (which should go really away on any
modern distro) and symlinks.

Well, we do not really "rely" on this. Only a small subset of devices is
exposed in systemd. Block devices, network devices, ttys, sound cards
and a few others. All of these have in common that they are not
particularly useful before they have been probed.

The device nodes in devtmpfs that the kernel passes to us can be used
without dep config in systemd. However, it's probably a good idea to
limit use to the static devices with this, and only those. (with other
words: it's probably a bad idea to use /dev/sdc7 before it has been
probed, but using /dev/null is completely fine, and it wouldn't show up
in systemd anyway).

Post by Scott James Remnant
There are probably many systems out there (hi!) that care so little about
symlinks that they're happy for udevd to be started non-critically in the
boot, and consider the "udevadm trigger" run a necessity that's probably the
last thing to run.

Again, the emphasis is on parallelization: we want to run everything in
parallel, no artificial ordering in userspace.

udev isn't just about symlinks. It's about perms and primarily about
meta data, and a lot of other stuff too.

Post by Scott James Remnant
What are the long term plans here? Do you envision this staying the way it
is?

I am sure Kay can provide you with a bit of interesting data that shows
you that udev git + systemd git are so quick enumerating devices that we
have a lot of other stuff that needs more optimization than udev right
now.

That all said we actually want to optimize udev further, and reduce the
number of rules we have to process. But that's more Kay's turf, so ping
him if you have questions.

Lennart

--
Lennart Poettering - Red Hat, Inc.

Koen Kooi

2011-05-09 19:51:18 UTC

Correct.

That's actually dead fast

It's still in the 10 second range on a 600MHz cortex-a8 machine booting from an SD card. I need to dig out my 400MHz arm920t to see how long it takes there. So having udev-less operation in systemd would be nice, even if it's only used on 'embedded'

regards,

Koen

Lennart Poettering

2011-05-09 20:02:56 UTC

That's actually dead fast

It's still in the 10 second range on a 600MHz cortex-a8 machine
booting from an SD card. I need to dig out my 400MHz arm920t to see
how long it takes there. So having udev-less operation in systemd
would be nice, even if it's only used on 'embedded'

Have you checked what precisely takes so long? My guess is that some
driver has a slow sysfs uevent trigger callback, and that might be
fixable in the driver. You might want to use strace with timestamps to
figure out why udev trigger might be so slow.

Lennart

--
Lennart Poettering - Red Hat, Inc.

Lennart Poettering

2011-05-09 20:08:46 UTC

Post by Koen Kooi
It's still in the 10 second range on a 600MHz cortex-a8 machine
booting from an SD card. I need to dig out my 400MHz arm920t to see
how long it takes there. So having udev-less operation in systemd
would be nice, even if it's only used on 'embedded'

Also note that only very few devices are exposed and depended on by
systemd. In an embedded env you might be able to simple trigger those
explicitly first, and then trigger the rest afterwards. That way you
don't have to wait 10s, and the triggering will be dispatched in the bg,
but you still get the flexibility that udev can handle hotplugged
devices.

Kay, opinions?

Lennart

--
Lennart Poettering - Red Hat, Inc.

Kay Sievers

2011-05-09 20:23:53 UTC

Guess that should work fine. I guess we could short-cut some stuff too
if it helps for custom setups, so systemd would look if the device is
already there, instead of wait for the tag to show up. But we would
need to find out if that really helps.

Real numbers of the most recent udev on such systems would help, so we
get an idea what we are dealing with.

Things like:
time (udevadm trigger; udevadm settle)
would be good to know too. If that looks like something to fix, the output of:
udevadm monitor
during the trigger command above will be useful too.

Kay

Koen Kooi

2011-05-10 08:14:40 UTC

Here's a systemd-analyze plot with udev 168: http://dominion.thruhere.net/koen/angstrom/systemd/168.svg

That's run on a beagleboard C3 running at 500Mhz on a 2GB sandisk SD card (±10MB/s read/write)

Post by Kay Sievers
Real numbers of the most recent udev on such systems would help, so we
get an idea what we are dealing with.
time (udevadm trigger; udevadm settle)
udevadm monitor
during the trigger command above will be useful too.

I'll do that later today.

regards,

Koen

Koen Kooi

2011-05-10 08:26:24 UTC

Guess that should work fine. I guess we could short-cut some stuff too
if it helps for custom setups, so systemd would look if the device is
already there, instead of wait for the tag to show up. But we would
need to find out if that really helps.
Real numbers of the most recent udev on such systems would help, so we
get an idea what we are dealing with.
time (udevadm trigger; udevadm settle)

***@beagleboard-systemd:~# time ( udevadm trigger ; udevadm settle)

real 3m0.475s
user 0m0.008s
sys 0m0.117s

Post by Kay Sievers
udevadm monitor

monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent

KERNEL[1305015564.600250] change /devices/omapdss/display0 (omapdss)
KERNEL[1305015564.600768] change /devices/omapdss/display1 (omapdss)
KERNEL[1305015564.601165] change /devices/platform/arm-pmu.0 (platform)
KERNEL[1305015564.601501] change /devices/platform/ehci-omap.0 (platform)
KERNEL[1305015564.601959] change /devices/platform/ehci-omap.0/usb1 (usb)
KERNEL[1305015564.602386] change /devices/platform/ehci-omap.0/usb1/1-0:1.0 (usb)
KERNEL[1305015564.602813] change /devices/platform/ehci-omap.0/usb1/1-2 (usb)
KERNEL[1305015564.603240] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1 (usb)
KERNEL[1305015564.603698] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0 (usb)
KERNEL[1305015564.604125] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0/0003:04F2:0402.0001 (hid)
KERNEL[1305015564.604736] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0/input/input2 (input)
KERNEL[1305015564.605163] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0/input/input2/event2 (input)
KERNEL[1305015564.606628] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1 (usb)
KERNEL[1305015564.607177] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1/0003:04F2:0402.0002 (hid)
KERNEL[1305015564.608245] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1/input/input3 (input)
UDEV [1305015564.609832] change /devices/omapdss/display0 (omapdss)
KERNEL[1305015564.610351] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1/input/input3/event3 (input)
KERNEL[1305015564.610778] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/usb_device/usbdev1.3 (usb_device)
KERNEL[1305015564.612091] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2 (usb)
KERNEL[1305015564.614013] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0 (usb)
UDEV [1305015564.614166] change /devices/omapdss/display1 (omapdss)
KERNEL[1305015564.614654] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/0003:046D:C00E.0003 (hid)
KERNEL[1305015564.615661] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input4 (input)
KERNEL[1305015564.616729] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input4/event4 (input)
KERNEL[1305015564.619232] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input4/mouse0 (input)
KERNEL[1305015564.619781] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/usb_device/usbdev1.4 (usb_device)
KERNEL[1305015564.620269] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.4 (usb)
KERNEL[1305015564.620727] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.4/1-2.4:1.0 (usb)
KERNEL[1305015564.621978] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.4/usb_device/usbdev1.5 (usb_device)
KERNEL[1305015564.623962] change /devices/platform/ehci-omap.0/usb1/1-2/1-2:1.0 (usb)
KERNEL[1305015564.624450] change /devices/platform/ehci-omap.0/usb1/1-2/usb_device/usbdev1.2 (usb_device)
KERNEL[1305015564.624847] change /devices/platform/ehci-omap.0/usb1/usb_device/usbdev1.1 (usb_device)
KERNEL[1305015564.626129] change /devices/platform/ehci-omap.0/usbmon/usbmon1 (usbmon)
UDEV [1305015564.626251] change /devices/platform/ehci-omap.0 (platform)
KERNEL[1305015564.626647] change /devices/platform/gpio-keys (platform)
KERNEL[1305015564.627654] change /devices/platform/gpio-keys/input/input0 (input)
KERNEL[1305015564.630248] change /devices/platform/gpio-keys/input/input0/event0 (input)
KERNEL[1305015564.630737] change /devices/platform/leds-gpio (platform)
KERNEL[1305015564.631225] change /devices/platform/leds-gpio/leds/beagleboard::pmu_stat (leds)
KERNEL[1305015564.631591] change /devices/platform/leds-gpio/leds/beagleboard::usr0 (leds)
KERNEL[1305015564.631896] change /devices/platform/leds-gpio/leds/beagleboard::usr1 (leds)
KERNEL[1305015564.639129] change /devices/platform/mmci-omap-hs.0 (platform)
KERNEL[1305015564.639648] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0 (mmc_host)
KERNEL[1305015564.644775] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa (mmc)
KERNEL[1305015564.644958] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa/block/mmcblk0 (block)
KERNEL[1305015564.645111] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa/block/mmcblk0/mmcblk0p1 (block)
KERNEL[1305015564.645233] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa/block/mmcblk0/mmcblk0p2 (block)
KERNEL[1305015564.645385] change /devices/platform/mmci-omap-hs.1 (platform)
KERNEL[1305015564.645477] change /devices/platform/mmci-omap-hs.1/mmc_host/mmc1 (mmc_host)
KERNEL[1305015564.645599] change /devices/platform/musb_hdrc (platform)
KERNEL[1305015564.645690] change /devices/platform/omap-aes (platform)
KERNEL[1305015564.645812] change /devices/platform/omap-iommu.0 (platform)
KERNEL[1305015564.645935] change /devices/platform/omap-mailbox (platform)
KERNEL[1305015564.646026] change /devices/platform/omap-mcbsp-dai.0 (platform)
KERNEL[1305015564.646148] change /devices/platform/omap-mcbsp-dai.1 (platform)
KERNEL[1305015564.646270] change /devices/platform/omap-mcbsp-dai.2 (platform)
KERNEL[1305015564.646392] change /devices/platform/omap-mcbsp-dai.3 (platform)
KERNEL[1305015564.646545] change /devices/platform/omap-mcbsp-dai.4 (platform)
KERNEL[1305015564.646758] change /devices/platform/omap-mcbsp.1 (platform)
KERNEL[1305015564.646911] change /devices/platform/omap-mcbsp.2 (platform)
KERNEL[1305015564.647003] change /devices/platform/omap-mcbsp.3 (platform)
KERNEL[1305015564.652587] change /devices/platform/omap-mcbsp.4 (platform)
KERNEL[1305015564.652770] change /devices/platform/omap-mcbsp.5 (platform)
KERNEL[1305015564.652893] change /devices/platform/omap-pcm-audio (platform)
KERNEL[1305015564.654724] change /devices/platform/omap-sham (platform)
KERNEL[1305015564.654846] change /devices/platform/omap/iva.0 (platform)
KERNEL[1305015564.654968] change /devices/platform/omap/l3_main.0 (platform)
KERNEL[1305015564.655059] change /devices/platform/omap/mpu.0 (platform)
KERNEL[1305015564.655181] change /devices/platform/omap/omap_dma_system.0 (platform)
KERNEL[1305015564.655303] change /devices/platform/omap/omap_gpio.0 (platform)
KERNEL[1305015564.655425] change /devices/platform/omap/omap_gpio.1 (platform)
KERNEL[1305015564.655517] change /devices/platform/omap/omap_gpio.2 (platform)
KERNEL[1305015564.655639] change /devices/platform/omap/omap_gpio.3 (platform)
KERNEL[1305015564.655761] change /devices/platform/omap/omap_gpio.4 (platform)
KERNEL[1305015564.670440] change /devices/platform/omap/omap_gpio.5 (platform)
UDEV [1305015564.670593] change /devices/platform/ehci-omap.0/usbmon/usbmon1 (usbmon)
KERNEL[1305015564.670715] change /devices/platform/omap/omap_i2c.1 (platform)
KERNEL[1305015564.670837] change /devices/platform/omap/omap_i2c.1/i2c-1 (i2c)
KERNEL[1305015564.670959] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0048 (i2c)
KERNEL[1305015564.671081] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0048/twl4030_usb (platform)
KERNEL[1305015564.671203] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049 (i2c)
KERNEL[1305015564.671325] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030-audio (platform)
KERNEL[1305015564.671447] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030-audio/twl4030-codec (platform)
KERNEL[1305015564.671569] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_gpio (platform)
KERNEL[1305015564.671691] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_gpio/gpio/gpiochip192 (gpio)
KERNEL[1305015564.671813] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_pwrbutton (platform)
KERNEL[1305015564.671935] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_pwrbutton/input/input1 (input)
KERNEL[1305015564.672088] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_pwrbutton/input/input1/event1 (input)
KERNEL[1305015564.672485] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004a (i2c)
KERNEL[1305015564.672607] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004a/twl4030_madc (platform)
KERNEL[1305015564.672729] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b (i2c)
KERNEL[1305015564.672851] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.17 (platform)
KERNEL[1305015564.672973] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.17/regulator/regulator.1 (regulator)
KERNEL[1305015564.673095] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.18 (platform)
KERNEL[1305015564.673217] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.18/regulator/regulator.2 (regulator)
KERNEL[1305015564.673309] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.19 (platform)
KERNEL[1305015564.673431] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.19/regulator/regulator.3 (regulator)
KERNEL[1305015564.673553] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.3 (platform)
KERNEL[1305015564.673644] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.3/regulator/regulator.5 (regulator)
KERNEL[1305015564.673767] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.5 (platform)
KERNEL[1305015564.673858] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.5/regulator/regulator.6 (regulator)
KERNEL[1305015564.673980] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.6 (platform)
KERNEL[1305015564.674102] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.6/regulator/regulator.4 (regulator)
KERNEL[1305015564.674194] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.8 (platform)
KERNEL[1305015564.674316] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.8/regulator/regulator.7 (regulator)
KERNEL[1305015564.674438] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_rtc (platform)
KERNEL[1305015564.674560] change /devices/platform/omap/omap_i2c.1/i2c-1/i2c-dev/i2c-1 (i2c-dev)
KERNEL[1305015564.674682] change /devices/platform/omap/omap_i2c.2 (platform)
KERNEL[1305015564.674774] change /devices/platform/omap/omap_i2c.2/i2c-2 (i2c)
KERNEL[1305015564.674896] change /devices/platform/omap/omap_i2c.2/i2c-2/2-0068 (i2c)
KERNEL[1305015564.675018] change /devices/platform/omap/omap_i2c.2/i2c-2/2-0068/rtc/rtc0 (rtc)
KERNEL[1305015564.675140] change /devices/platform/omap/omap_i2c.2/i2c-2/i2c-dev/i2c-2 (i2c-dev)
KERNEL[1305015564.675262] change /devices/platform/omap/omap_i2c.3 (platform)
KERNEL[1305015564.675353] change /devices/platform/omap/omap_i2c.3/i2c-3 (i2c)
KERNEL[1305015564.675476] change /devices/platform/omap/omap_i2c.3/i2c-3/3-0050 (i2c)
KERNEL[1305015564.675598] change /devices/platform/omap/omap_i2c.3/i2c-3/i2c-dev/i2c-3 (i2c-dev)
KERNEL[1305015564.675720] change /devices/platform/omap/omap_uart.0 (platform)
UDEV [1305015564.678680] change /devices/platform/gpio-keys (platform)
KERNEL[1305015564.687194] change /devices/platform/omap/omap_uart.0/tty/ttyO0 (tty)
KERNEL[1305015564.687347] change /devices/platform/omap/omap_uart.1 (platform)
KERNEL[1305015564.687469] change /devices/platform/omap/omap_uart.1/tty/ttyO1 (tty)
KERNEL[1305015564.687591] change /devices/platform/omap/omap_uart.2 (platform)
KERNEL[1305015564.687713] change /devices/platform/omap/omap_uart.2/tty/ttyO2 (tty)
KERNEL[1305015564.687835] change /devices/platform/omap/omap_wdt (platform)
KERNEL[1305015564.687957] change /devices/platform/omap/omap_wdt/misc/watchdog (misc)
KERNEL[1305015564.688079] change /devices/platform/omap/smartreflex.0 (platform)
KERNEL[1305015564.688201] change /devices/platform/omap/smartreflex.1 (platform)
KERNEL[1305015564.688323] change /devices/platform/omap2-nand.0 (platform)
KERNEL[1305015564.688445] change /devices/platform/omap2_mcspi.1 (platform)
KERNEL[1305015564.688537] change /devices/platform/omap2_mcspi.1/spi_master/spi1 (spi_master)
KERNEL[1305015564.688659] change /devices/platform/omap2_mcspi.2 (platform)
KERNEL[1305015564.689025] change /devices/platform/omap2_mcspi.2/spi_master/spi2 (spi_master)
KERNEL[1305015564.689147] change /devices/platform/omap2_mcspi.3 (platform)
KERNEL[1305015564.689239] change /devices/platform/omap2_mcspi.3/spi_master/spi3 (spi_master)
KERNEL[1305015564.689361] change /devices/platform/omap2_mcspi.4 (platform)
KERNEL[1305015564.689483] change /devices/platform/omap2_mcspi.4/spi4.0 (spi)
KERNEL[1305015564.689605] change /devices/platform/omap2_mcspi.4/spi4.0/net/eth0 (net)
KERNEL[1305015564.689697] change /devices/platform/omap2_mcspi.4/spi_master/spi4 (spi_master)
KERNEL[1305015564.689819] change /devices/platform/omap_vout (platform)
KERNEL[1305015564.689941] change /devices/platform/omap_vout/video4linux/video1 (video4linux)
KERNEL[1305015564.690063] change /devices/platform/omapdss (platform)
KERNEL[1305015564.690185] change /devices/platform/omapfb (platform)
KERNEL[1305015564.690307] change /devices/platform/omapfb/graphics/fb0 (graphics)
KERNEL[1305015564.690429] change /devices/platform/omapfb/graphics/fb1 (graphics)
KERNEL[1305015564.690521] change /devices/platform/reg-dummy (platform)
KERNEL[1305015564.701812] change /devices/platform/serial8250 (platform)
KERNEL[1305015564.702331] change /devices/platform/serial8250/tty/ttyS0 (tty)
KERNEL[1305015564.702728] change /devices/platform/serial8250/tty/ttyS1 (tty)
KERNEL[1305015564.703094] change /devices/platform/serial8250/tty/ttyS2 (tty)
KERNEL[1305015564.703430] change /devices/platform/serial8250/tty/ttyS3 (tty)
KERNEL[1305015564.703765] change /devices/platform/soc-audio (platform)
KERNEL[1305015564.704101] change /devices/platform/soc-audio/sound/card0 (sound)
KERNEL[1305015564.704467] change /devices/platform/soc-audio/sound/card0/audio (sound)
KERNEL[1305015564.704803] change /devices/platform/soc-audio/sound/card0/dsp (sound)
KERNEL[1305015564.705139] change /devices/platform/soc-audio/sound/card0/mixer (sound)
KERNEL[1305015564.705505] change /devices/platform/soc-audio/sound/card0/pcmC0D0c (sound)
KERNEL[1305015564.705871] change /devices/platform/soc-audio/sound/card0/pcmC0D0p (sound)
KERNEL[1305015564.706237] change /devices/platform/soc-audio/sound/card0/controlC0 (sound)
KERNEL[1305015564.706542] change /devices/virtual/bdi/179:0 (bdi)
KERNEL[1305015564.706878] change /devices/virtual/bdi/1:0 (bdi)
KERNEL[1305015564.707183] change /devices/virtual/bdi/1:1 (bdi)
KERNEL[1305015564.707488] change /devices/virtual/bdi/1:10 (bdi)
KERNEL[1305015564.707794] change /devices/virtual/bdi/1:11 (bdi)
KERNEL[1305015564.708099] change /devices/virtual/bdi/1:12 (bdi)
KERNEL[1305015564.708404] change /devices/virtual/bdi/1:13 (bdi)
KERNEL[1305015564.708709] change /devices/virtual/bdi/1:14 (bdi)
KERNEL[1305015564.709045] change /devices/virtual/bdi/1:15 (bdi)
KERNEL[1305015564.710601] change /devices/virtual/bdi/1:2 (bdi)
KERNEL[1305015564.710967] change /devices/virtual/bdi/1:3 (bdi)
UDEV [1305015564.712066] change /devices/platform/arm-pmu.0 (platform)
KERNEL[1305015564.712463] change /devices/virtual/bdi/1:4 (bdi)
KERNEL[1305015564.712829] change /devices/virtual/bdi/1:5 (bdi)
KERNEL[1305015564.714233] change /devices/virtual/bdi/1:6 (bdi)
KERNEL[1305015564.714569] change /devices/virtual/bdi/1:7 (bdi)
KERNEL[1305015564.716491] change /devices/virtual/bdi/1:8 (bdi)
UDEV [1305015564.716644] change /devices/platform/leds-gpio (platform)
KERNEL[1305015564.717620] change /devices/virtual/bdi/1:9 (bdi)
KERNEL[1305015564.732116] change /devices/virtual/bdi/31:0 (bdi)
UDEV [1305015564.732238] change /devices/platform/leds-gpio/leds/beagleboard::pmu_stat (leds)
KERNEL[1305015564.732391] change /devices/virtual/bdi/31:1 (bdi)
UDEV [1305015564.732513] change /devices/platform/leds-gpio/leds/beagleboard::usr0 (leds)
KERNEL[1305015564.735626] change /devices/virtual/bdi/31:2 (bdi)
UDEV [1305015564.735931] change /devices/platform/ehci-omap.0/usb1 (usb)
KERNEL[1305015564.736083] change /devices/virtual/bdi/31:3 (bdi)
KERNEL[1305015564.736450] change /devices/virtual/bdi/31:4 (bdi)
KERNEL[1305015564.736541] change /devices/virtual/bdi/7:0 (bdi)
KERNEL[1305015564.736663] change /devices/virtual/bdi/7:1 (bdi)
KERNEL[1305015564.736755] change /devices/virtual/bdi/7:2 (bdi)
KERNEL[1305015564.736877] change /devices/virtual/bdi/7:3 (bdi)
KERNEL[1305015564.736968] change /devices/virtual/bdi/7:4 (bdi)
KERNEL[1305015564.737060] change /devices/virtual/bdi/7:5 (bdi)
KERNEL[1305015564.737152] change /devices/virtual/bdi/7:6 (bdi)
KERNEL[1305015564.737274] change /devices/virtual/bdi/7:7 (bdi)
KERNEL[1305015564.737365] change /devices/virtual/bdi/default (bdi)
KERNEL[1305015564.737457] change /devices/virtual/bdi/mtd-romap (bdi)
KERNEL[1305015564.737579] change /devices/virtual/bdi/mtd-rwmap (bdi)
KERNEL[1305015564.737670] change /devices/virtual/bdi/mtd-unmap (bdi)
KERNEL[1305015564.737792] change /devices/virtual/block/loop0 (block)
KERNEL[1305015564.737914] change /devices/virtual/block/loop1 (block)
KERNEL[1305015564.738037] change /devices/virtual/block/loop2 (block)
KERNEL[1305015564.738159] change /devices/virtual/block/loop3 (block)
KERNEL[1305015564.738311] change /devices/virtual/block/loop4 (block)
KERNEL[1305015564.738433] change /devices/virtual/block/loop5 (block)
KERNEL[1305015564.738555] change /devices/virtual/block/loop6 (block)
KERNEL[1305015564.738677] change /devices/virtual/block/loop7 (block)
KERNEL[1305015564.738800] change /devices/virtual/block/ram0 (block)
KERNEL[1305015564.738922] change /devices/virtual/block/ram1 (block)
KERNEL[1305015564.739074] change /devices/virtual/block/ram10 (block)
KERNEL[1305015564.739196] change /devices/virtual/block/ram11 (block)
KERNEL[1305015564.739318] change /devices/virtual/block/ram12 (block)
KERNEL[1305015564.739440] change /devices/virtual/block/ram13 (block)
KERNEL[1305015564.739562] change /devices/virtual/block/ram14 (block)
KERNEL[1305015564.739715] change /devices/virtual/block/ram15 (block)
KERNEL[1305015564.739837] change /devices/virtual/block/ram2 (block)
KERNEL[1305015564.739959] change /devices/virtual/block/ram3 (block)
KERNEL[1305015564.740081] change /devices/virtual/block/ram4 (block)
KERNEL[1305015564.740234] change /devices/virtual/block/ram5 (block)
KERNEL[1305015564.740386] change /devices/virtual/block/ram6 (block)
KERNEL[1305015564.745605] change /devices/virtual/block/ram7 (block)
UDEV [1305015564.745758] change /devices/platform/ehci-omap.0/usb1/1-0:1.0 (usb)
KERNEL[1305015564.745910] change /devices/virtual/block/ram8 (block)
UDEV [1305015564.746032] change /devices/platform/ehci-omap.0/usb1/usb_device/usbdev1.1 (usb_device)
KERNEL[1305015564.746154] change /devices/virtual/block/ram9 (block)
KERNEL[1305015564.746276] change /devices/virtual/gpio/gpio157 (gpio)
KERNEL[1305015564.746368] change /devices/virtual/gpio/gpiochip0 (gpio)
KERNEL[1305015564.746490] change /devices/virtual/gpio/gpiochip128 (gpio)
KERNEL[1305015564.746581] change /devices/virtual/gpio/gpiochip160 (gpio)
KERNEL[1305015564.746673] change /devices/virtual/gpio/gpiochip32 (gpio)
KERNEL[1305015564.746795] change /devices/virtual/gpio/gpiochip64 (gpio)
KERNEL[1305015564.746887] change /devices/virtual/gpio/gpiochip96 (gpio)
KERNEL[1305015564.746978] change /devices/virtual/graphics/fbcon (graphics)
KERNEL[1305015564.747100] change /devices/virtual/input/mice (input)
KERNEL[1305015564.747222] change /devices/virtual/mem/full (mem)
KERNEL[1305015564.747344] change /devices/virtual/mem/kmem (mem)
KERNEL[1305015564.764221] change /devices/virtual/mem/kmsg (mem)
UDEV [1305015564.764373] change /devices/platform/leds-gpio/leds/beagleboard::usr1 (leds)
KERNEL[1305015564.764495] change /devices/virtual/mem/mem (mem)
UDEV [1305015564.764617] change /devices/platform/mmci-omap-hs.0 (platform)
KERNEL[1305015564.764739] change /devices/virtual/mem/null (mem)
KERNEL[1305015564.764892] change /devices/virtual/mem/random (mem)
KERNEL[1305015564.765014] change /devices/virtual/mem/urandom (mem)
KERNEL[1305015564.765411] change /devices/virtual/mem/zero (mem)
KERNEL[1305015564.765533] change /devices/virtual/misc/autofs (misc)
KERNEL[1305015564.765655] change /devices/virtual/misc/cpu_dma_latency (misc)
KERNEL[1305015564.765777] change /devices/virtual/misc/network_latency (misc)
KERNEL[1305015564.765930] change /devices/virtual/misc/network_throughput (misc)
KERNEL[1305015564.766052] change /devices/virtual/misc/psaux (misc)
KERNEL[1305015564.766174] change /devices/virtual/misc/twl4030-madc (misc)
KERNEL[1305015564.766296] change /devices/virtual/misc/ubi_ctrl (misc)
KERNEL[1305015564.766418] change /devices/virtual/misc/uinput (misc)
KERNEL[1305015564.766540] change /devices/virtual/mtd/mtd0 (mtd)
KERNEL[1305015564.766662] change /devices/virtual/mtd/mtd0/mtdblock0 (block)
KERNEL[1305015564.766784] change /devices/virtual/mtd/mtd0ro (mtd)
KERNEL[1305015564.766906] change /devices/virtual/mtd/mtd1 (mtd)
KERNEL[1305015564.767059] change /devices/virtual/mtd/mtd1/mtdblock1 (block)
KERNEL[1305015564.767181] change /devices/virtual/mtd/mtd1ro (mtd)
KERNEL[1305015564.767303] change /devices/virtual/mtd/mtd2 (mtd)
KERNEL[1305015564.767425] change /devices/virtual/mtd/mtd2/mtdblock2 (block)
KERNEL[1305015564.767547] change /devices/virtual/mtd/mtd2ro (mtd)
KERNEL[1305015564.767669] change /devices/virtual/mtd/mtd3 (mtd)
KERNEL[1305015564.767791] change /devices/virtual/mtd/mtd3/mtdblock3 (block)
KERNEL[1305015564.767913] change /devices/virtual/mtd/mtd3ro (mtd)
KERNEL[1305015564.768066] change /devices/virtual/mtd/mtd4 (mtd)
KERNEL[1305015564.768188] change /devices/virtual/mtd/mtd4/mtdblock4 (block)
KERNEL[1305015564.768310] change /devices/virtual/mtd/mtd4ro (mtd)
KERNEL[1305015564.768432] change /devices/virtual/net/lo (net)
KERNEL[1305015564.768524] change /devices/virtual/regulator/regulator.0 (regulator)
KERNEL[1305015564.768646] change /devices/virtual/sound/timer (sound)
KERNEL[1305015564.768768] change /devices/virtual/tty/console (tty)
KERNEL[1305015564.768890] change /devices/virtual/tty/ptmx (tty)
KERNEL[1305015564.769012] change /devices/virtual/tty/tty (tty)
KERNEL[1305015564.769164] change /devices/virtual/tty/tty0 (tty)
KERNEL[1305015564.769256] change /devices/virtual/tty/tty1 (tty)
KERNEL[1305015564.769378] change /devices/virtual/tty/tty10 (tty)
KERNEL[1305015564.769500] change /devices/virtual/tty/tty11 (tty)
KERNEL[1305015564.769622] change /devices/virtual/tty/tty12 (tty)
KERNEL[1305015564.769744] change /devices/virtual/tty/tty13 (tty)
KERNEL[1305015564.769866] change /devices/virtual/tty/tty14 (tty)
KERNEL[1305015564.769988] change /devices/virtual/tty/tty15 (tty)
KERNEL[1305015564.770111] change /devices/virtual/tty/tty16 (tty)
KERNEL[1305015564.770233] change /devices/virtual/tty/tty17 (tty)
KERNEL[1305015564.770355] change /devices/virtual/tty/tty18 (tty)
KERNEL[1305015564.770477] change /devices/virtual/tty/tty19 (tty)
KERNEL[1305015564.770599] change /devices/virtual/tty/tty2 (tty)
KERNEL[1305015564.770721] change /devices/virtual/tty/tty20 (tty)
KERNEL[1305015564.770843] change /devices/virtual/tty/tty21 (tty)
KERNEL[1305015564.770965] change /devices/virtual/tty/tty22 (tty)
KERNEL[1305015564.771087] change /devices/virtual/tty/tty23 (tty)
KERNEL[1305015564.771209] change /devices/virtual/tty/tty24 (tty)
KERNEL[1305015564.771331] change /devices/virtual/tty/tty25 (tty)
UDEV [1305015564.795288] change /devices/platform/mmci-omap-hs.1 (platform)
UDEV [1305015564.795471] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0 (mmc_host)
KERNEL[1305015564.795593] change /devices/virtual/tty/tty26 (tty)
UDEV [1305015564.795745] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa (mmc)
KERNEL[1305015564.795867] change /devices/virtual/tty/tty27 (tty)
UDEV [1305015564.796020] change /devices/platform/gpio-keys/input/input0 (input)
KERNEL[1305015564.796142] change /devices/virtual/tty/tty28 (tty)
KERNEL[1305015564.796264] change /devices/virtual/tty/tty29 (tty)
KERNEL[1305015564.796661] change /devices/virtual/tty/tty3 (tty)
KERNEL[1305015564.796813] change /devices/virtual/tty/tty30 (tty)
KERNEL[1305015564.796935] change /devices/virtual/tty/tty31 (tty)
KERNEL[1305015564.797058] change /devices/virtual/tty/tty32 (tty)
KERNEL[1305015564.797180] change /devices/virtual/tty/tty33 (tty)
KERNEL[1305015564.797302] change /devices/virtual/tty/tty34 (tty)
KERNEL[1305015564.797393] change /devices/virtual/tty/tty35 (tty)
KERNEL[1305015564.797515] change /devices/virtual/tty/tty36 (tty)
KERNEL[1305015564.797637] change /devices/virtual/tty/tty37 (tty)
KERNEL[1305015564.797759] change /devices/virtual/tty/tty38 (tty)
KERNEL[1305015564.797882] change /devices/virtual/tty/tty39 (tty)
KERNEL[1305015564.798004] change /devices/virtual/tty/tty4 (tty)
KERNEL[1305015564.798126] change /devices/virtual/tty/tty40 (tty)
KERNEL[1305015564.798248] change /devices/virtual/tty/tty41 (tty)
KERNEL[1305015564.798370] change /devices/virtual/tty/tty42 (tty)
KERNEL[1305015564.798492] change /devices/virtual/tty/tty43 (tty)
KERNEL[1305015564.798614] change /devices/virtual/tty/tty44 (tty)
KERNEL[1305015564.798736] change /devices/virtual/tty/tty45 (tty)
KERNEL[1305015564.798858] change /devices/virtual/tty/tty46 (tty)
KERNEL[1305015564.798980] change /devices/virtual/tty/tty47 (tty)
KERNEL[1305015564.799102] change /devices/virtual/tty/tty48 (tty)
KERNEL[1305015564.799224] change /devices/virtual/tty/tty49 (tty)
KERNEL[1305015564.799346] change /devices/virtual/tty/tty5 (tty)
KERNEL[1305015564.799468] change /devices/virtual/tty/tty50 (tty)
KERNEL[1305015564.799591] change /devices/virtual/tty/tty51 (tty)
KERNEL[1305015564.799713] change /devices/virtual/tty/tty52 (tty)
KERNEL[1305015564.799835] change /devices/virtual/tty/tty53 (tty)
KERNEL[1305015564.799957] change /devices/virtual/tty/tty54 (tty)
KERNEL[1305015564.800079] change /devices/virtual/tty/tty55 (tty)
KERNEL[1305015564.800201] change /devices/virtual/tty/tty56 (tty)
KERNEL[1305015564.800323] change /devices/virtual/tty/tty57 (tty)
KERNEL[1305015564.800445] change /devices/virtual/tty/tty58 (tty)
KERNEL[1305015564.800567] change /devices/virtual/tty/tty59 (tty)
KERNEL[1305015564.800689] change /devices/virtual/tty/tty6 (tty)
KERNEL[1305015564.800811] change /devices/virtual/tty/tty60 (tty)
KERNEL[1305015564.800933] change /devices/virtual/tty/tty61 (tty)
KERNEL[1305015564.801055] change /devices/virtual/tty/tty62 (tty)
KERNEL[1305015564.801177] change /devices/virtual/tty/tty63 (tty)
KERNEL[1305015564.801300] change /devices/virtual/tty/tty7 (tty)
KERNEL[1305015564.801422] change /devices/virtual/tty/tty8 (tty)
KERNEL[1305015564.801544] change /devices/virtual/tty/tty9 (tty)
KERNEL[1305015564.801666] change /devices/virtual/tty/ttyprintk (tty)
KERNEL[1305015564.801788] change /devices/virtual/usbmon/usbmon0 (usbmon)
KERNEL[1305015564.801910] change /devices/virtual/vc/vcs (vc)
KERNEL[1305015564.802032] change /devices/virtual/vc/vcs1 (vc)
KERNEL[1305015564.802154] change /devices/virtual/vc/vcs2 (vc)
KERNEL[1305015564.802276] change /devices/virtual/vc/vcs3 (vc)
KERNEL[1305015564.802398] change /devices/virtual/vc/vcs4 (vc)
KERNEL[1305015564.802490] change /devices/virtual/vc/vcs5 (vc)
KERNEL[1305015564.802612] change /devices/virtual/vc/vcs6 (vc)
KERNEL[1305015564.802795] change /devices/virtual/vc/vcsa (vc)
KERNEL[1305015564.813507] change /devices/virtual/vc/vcsa1 (vc)
KERNEL[1305015564.813720] change /devices/virtual/vc/vcsa2 (vc)
UDEV [1305015564.813842] change /devices/platform/mmci-omap-hs.1/mmc_host/mmc1 (mmc_host)
KERNEL[1305015564.813964] change /devices/virtual/vc/vcsa3 (vc)
UDEV [1305015564.814056] change /devices/platform/musb_hdrc (platform)
KERNEL[1305015564.826660] change /devices/virtual/vc/vcsa4 (vc)
KERNEL[1305015564.826843] change /devices/virtual/vc/vcsa5 (vc)
KERNEL[1305015564.826965] change /devices/virtual/vc/vcsa6 (vc)
KERNEL[1305015564.827087] change /devices/virtual/vtconsole/vtcon0 (vtconsole)
KERNEL[1305015564.827453] change /devices/virtual/vtconsole/vtcon1 (vtconsole)
UDEV [1305015564.846679] change /devices/platform/omap-iommu.0 (platform)
UDEV [1305015564.856018] change /devices/platform/ehci-omap.0/usb1/1-2 (usb)
UDEV [1305015564.874694] change /devices/platform/ehci-omap.0/usb1/1-2/1-2:1.0 (usb)
UDEV [1305015564.878082] change /devices/platform/ehci-omap.0/usb1/1-2/usb_device/usbdev1.2 (usb_device)
UDEV [1305015564.880248] change /devices/platform/omap-mcbsp-dai.0 (platform)
UDEV [1305015564.894409] change /devices/platform/omap-aes (platform)
UDEV [1305015564.907531] change /devices/platform/omap-mcbsp-dai.1 (platform)
UDEV [1305015564.910278] change /devices/platform/omap-mcbsp-dai.2 (platform)
UDEV [1305015564.914825] change /devices/platform/omap-mcbsp-dai.3 (platform)
UDEV [1305015564.922820] change /devices/platform/omap-mcbsp-dai.4 (platform)
UDEV [1305015564.931243] change /devices/platform/omap-mcbsp.1 (platform)
UDEV [1305015564.933593] change /devices/platform/omap-mcbsp.2 (platform)
UDEV [1305015564.954254] change /devices/platform/omap-mcbsp.3 (platform)
UDEV [1305015564.956603] change /devices/platform/omap-mcbsp.4 (platform)
UDEV [1305015564.957885] change /devices/platform/omap-mailbox (platform)
UDEV [1305015564.964050] change /devices/platform/omap-mcbsp.5 (platform)
UDEV [1305015564.969665] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.4 (usb)
UDEV [1305015564.973083] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1 (usb)
UDEV [1305015564.975921] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2 (usb)
UDEV [1305015564.978057] change /devices/platform/omap-pcm-audio (platform)
UDEV [1305015564.981140] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0 (usb)
UDEV [1305015564.983550] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1 (usb)
UDEV [1305015564.986755] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/usb_device/usbdev1.3 (usb_device)
UDEV [1305015564.990661] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0 (usb)
UDEV [1305015564.994140] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/usb_device/usbdev1.4 (usb_device)
UDEV [1305015564.996246] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.4/1-2.4:1.0 (usb)
UDEV [1305015565.000091] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.4/usb_device/usbdev1.5 (usb_device)
UDEV [1305015565.006835] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0/0003:04F2:0402.0001 (hid)
UDEV [1305015565.022491] change /devices/platform/gpio-keys/input/input0/event0 (input)
UDEV [1305015565.035430] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1/0003:04F2:0402.0002 (hid)
UDEV [1305015565.047851] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/0003:046D:C00E.0003 (hid)
UDEV [1305015565.077331] change /devices/platform/omap/l3_main.0 (platform)
UDEV [1305015565.085021] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1/input/input3 (input)
UDEV [1305015565.101776] change /devices/platform/omap/omap_dma_system.0 (platform)
UDEV [1305015565.123107] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input4 (input)
UDEV [1305015565.129425] change /devices/platform/omap-sham (platform)
UDEV [1305015565.135284] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0/input/input2 (input)
UDEV [1305015565.156829] change /devices/platform/omap/omap_gpio.0 (platform)
UDEV [1305015565.210815] change /devices/platform/omap/omap_gpio.1 (platform)
UDEV [1305015565.233978] change /devices/platform/omap/mpu.0 (platform)
UDEV [1305015565.235778] change /devices/platform/omap/omap_gpio.2 (platform)
UDEV [1305015565.236358] change /devices/platform/omap/iva.0 (platform)
UDEV [1305015565.242523] change /devices/platform/omap/omap_gpio.3 (platform)
UDEV [1305015565.245208] change /devices/platform/omap/omap_gpio.4 (platform)
UDEV [1305015565.255279] change /devices/platform/omap/omap_gpio.5 (platform)
UDEV [1305015565.285461] change /devices/platform/omap/omap_i2c.1 (platform)
UDEV [1305015565.297668] change /devices/platform/omap/omap_i2c.1/i2c-1 (i2c)
UDEV [1305015565.314483] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049 (i2c)
UDEV [1305015565.316284] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0048 (i2c)
UDEV [1305015565.319518] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030-audio (platform)
UDEV [1305015565.321350] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0048/twl4030_usb (platform)
UDEV [1305015565.323883] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030-audio/twl4030-codec (platform)
UDEV [1305015565.328918] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_pwrbutton (platform)
UDEV [1305015565.334625] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_gpio (platform)
UDEV [1305015565.352050] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_gpio/gpio/gpiochip192 (gpio)
UDEV [1305015565.408813] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004a (i2c)
UDEV [1305015565.411376] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b (i2c)
UDEV [1305015565.446838] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.17 (platform)
UDEV [1305015565.462341] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004a/twl4030_madc (platform)
UDEV [1305015565.490203] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.17/regulator/regulator.1 (regulator)
UDEV [1305015565.567504] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.19 (platform)
UDEV [1305015565.569671] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.18 (platform)
UDEV [1305015565.571258] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.19/regulator/regulator.3 (regulator)
UDEV [1305015565.573577] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.18/regulator/regulator.2 (regulator)
UDEV [1305015565.591796] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.3 (platform)
UDEV [1305015565.620605] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_pwrbutton/input/input1 (input)
UDEV [1305015565.645172] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.3/regulator/regulator.5 (regulator)
UDEV [1305015565.649383] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.5 (platform)
UDEV [1305015565.670288] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.5/regulator/regulator.6 (regulator)
UDEV [1305015565.672973] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.6 (platform)
UDEV [1305015565.675537] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.8 (platform)
UDEV [1305015565.677825] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.8/regulator/regulator.7 (regulator)
UDEV [1305015565.687622] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_reg.6/regulator/regulator.4 (regulator)
UDEV [1305015565.691589] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.1/input/input3/event3 (input)
UDEV [1305015565.704772] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.1/1-2.1:1.0/input/input2/event2 (input)
UDEV [1305015565.717742] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input4/event4 (input)
UDEV [1305015565.733856] change /devices/platform/ehci-omap.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input4/mouse0 (input)
UDEV [1305015565.750976] change /devices/platform/omap/omap_i2c.1/i2c-1/i2c-dev/i2c-1 (i2c-dev)
UDEV [1305015565.771881] change /devices/platform/omap/omap_i2c.2 (platform)
UDEV [1305015565.775054] change /devices/platform/omap/omap_i2c.3 (platform)
UDEV [1305015565.777374] change /devices/platform/omap/omap_i2c.2/i2c-2 (i2c)
UDEV [1305015565.780181] change /devices/platform/omap/omap_i2c.3/i2c-3 (i2c)
UDEV [1305015565.793334] change /devices/platform/omap/omap_i2c.2/i2c-2/2-0068 (i2c)
UDEV [1305015565.813629] change /devices/platform/omap/omap_i2c.2/i2c-2/i2c-dev/i2c-2 (i2c-dev)
UDEV [1305015565.828155] change /devices/platform/omap/omap_i2c.2/i2c-2/2-0068/rtc/rtc0 (rtc)
UDEV [1305015565.850830] change /devices/platform/omap/omap_i2c.3/i2c-3/i2c-dev/i2c-3 (i2c-dev)
UDEV [1305015565.872894] change /devices/platform/omap/omap_uart.0 (platform)
UDEV [1305015565.892242] change /devices/platform/omap/omap_uart.0/tty/ttyO0 (tty)
UDEV [1305015565.894775] change /devices/platform/omap/omap_uart.1 (platform)
UDEV [1305015565.910064] change /devices/platform/omap/omap_uart.2 (platform)
UDEV [1305015565.915191] change /devices/platform/omap/omap_uart.1/tty/ttyO1 (tty)
UDEV [1305015565.920989] change /devices/platform/omap/omap_uart.2/tty/ttyO2 (tty)
UDEV [1305015565.929870] change /devices/platform/omap/omap_wdt (platform)
UDEV [1305015565.933898] change /devices/platform/omap/omap_wdt/misc/watchdog (misc)
UDEV [1305015565.946563] change /devices/platform/omap/omap_i2c.1/i2c-1/1-0049/twl4030_pwrbutton/input/input1/event1 (input)
UDEV [1305015565.955596] change /devices/platform/omap/smartreflex.0 (platform)
UDEV [1305015565.957519] change /devices/platform/omap/omap_i2c.3/i2c-3/3-0050 (i2c)
UDEV [1305015565.960937] change /devices/platform/omap/smartreflex.1 (platform)
UDEV [1305015565.996948] change /devices/platform/omap2-nand.0 (platform)
UDEV [1305015566.012084] change /devices/platform/omap2_mcspi.1 (platform)
UDEV [1305015566.014678] change /devices/platform/omap2_mcspi.2 (platform)
UDEV [1305015566.017028] change /devices/platform/omap2_mcspi.1/spi_master/spi1 (spi_master)
UDEV [1305015566.018676] change /devices/platform/omap2_mcspi.2/spi_master/spi2 (spi_master)
UDEV [1305015566.036102] change /devices/platform/omap2_mcspi.3 (platform)
UDEV [1305015566.036468] change /devices/platform/omap2_mcspi.4 (platform)
UDEV [1305015566.038940] change /devices/platform/omap2_mcspi.3/spi_master/spi3 (spi_master)
UDEV [1305015566.040954] change /devices/platform/omap2_mcspi.4/spi4.0 (spi)
UDEV [1305015566.044982] change /devices/platform/omap2_mcspi.4/spi4.0/net/eth0 (net)
UDEV [1305015566.047088] change /devices/platform/omap2_mcspi.4/spi_master/spi4 (spi_master)
UDEV [1305015566.055847] change /devices/platform/omap_vout (platform)
UDEV [1305015566.058288] change /devices/platform/omapdss (platform)
UDEV [1305015566.070953] change /devices/platform/omapfb (platform)
UDEV [1305015566.080932] change /devices/platform/omapfb/graphics/fb0 (graphics)
UDEV [1305015566.093566] change /devices/platform/omapfb/graphics/fb1 (graphics)
UDEV [1305015566.098388] change /devices/platform/omap/omap_i2c.1/i2c-1/1-004b/twl_rtc (platform)
UDEV [1305015566.107665] change /devices/platform/serial8250 (platform)
UDEV [1305015566.131683] change /devices/platform/soc-audio (platform)
UDEV [1305015566.144042] change /devices/platform/serial8250/tty/ttyS0 (tty)
UDEV [1305015566.156921] change /devices/platform/serial8250/tty/ttyS1 (tty)
UDEV [1305015566.179626] change /devices/platform/serial8250/tty/ttyS2 (tty)
UDEV [1305015566.183074] change /devices/platform/serial8250/tty/ttyS3 (tty)
UDEV [1305015566.197265] change /devices/virtual/bdi/179:0 (bdi)
UDEV [1305015566.216278] change /devices/virtual/bdi/1:0 (bdi)
UDEV [1305015566.218139] change /devices/platform/reg-dummy (platform)
UDEV [1305015566.220458] change /devices/virtual/bdi/1:1 (bdi)
UDEV [1305015566.226440] change /devices/virtual/bdi/1:10 (bdi)
UDEV [1305015566.235504] change /devices/virtual/bdi/1:12 (bdi)
UDEV [1305015566.236968] change /devices/virtual/bdi/1:11 (bdi)
UDEV [1305015566.255889] change /devices/virtual/bdi/1:14 (bdi)
UDEV [1305015566.257507] change /devices/virtual/bdi/1:13 (bdi)
UDEV [1305015566.260253] change /devices/virtual/bdi/1:15 (bdi)
UDEV [1305015566.263030] change /devices/platform/omap_vout/video4linux/video1 (video4linux)
UDEV [1305015566.279663] change /devices/virtual/bdi/1:2 (bdi)
UDEV [1305015566.281768] change /devices/virtual/bdi/1:3 (bdi)
UDEV [1305015566.284606] change /devices/virtual/bdi/1:4 (bdi)
UDEV [1305015566.286926] change /devices/platform/soc-audio/sound/card0 (sound)
UDEV [1305015566.297729] change /devices/platform/soc-audio/sound/card0/audio (sound)
UDEV [1305015566.301208] change /devices/platform/soc-audio/sound/card0/dsp (sound)
UDEV [1305015566.317626] change /devices/platform/soc-audio/sound/card0/mixer (sound)
UDEV [1305015566.321136] change /devices/platform/soc-audio/sound/card0/pcmC0D0c (sound)
UDEV [1305015566.324401] change /devices/platform/soc-audio/sound/card0/pcmC0D0p (sound)
UDEV [1305015566.336181] change /devices/virtual/bdi/1:5 (bdi)
UDEV [1305015566.338867] change /devices/virtual/bdi/1:6 (bdi)
UDEV [1305015566.356841] change /devices/virtual/bdi/1:7 (bdi)
UDEV [1305015566.359100] change /devices/virtual/bdi/1:8 (bdi)
UDEV [1305015566.361053] change /devices/virtual/bdi/1:9 (bdi)
UDEV [1305015566.362945] change /devices/virtual/bdi/31:0 (bdi)
UDEV [1305015566.374725] change /devices/virtual/bdi/31:2 (bdi)
UDEV [1305015566.376220] change /devices/virtual/bdi/31:1 (bdi)
UDEV [1305015566.378753] change /devices/virtual/bdi/31:4 (bdi)
UDEV [1305015566.386627] change /devices/virtual/bdi/31:3 (bdi)
UDEV [1305015566.391632] change /devices/virtual/bdi/7:1 (bdi)
UDEV [1305015566.400360] change /devices/virtual/bdi/7:0 (bdi)
UDEV [1305015566.403289] change /devices/virtual/bdi/7:3 (bdi)
UDEV [1305015566.428283] change /devices/virtual/bdi/7:4 (bdi)
UDEV [1305015566.428771] change /devices/virtual/bdi/7:2 (bdi)
UDEV [1305015566.431335] change /devices/virtual/bdi/7:6 (bdi)
UDEV [1305015566.432678] change /devices/virtual/bdi/7:5 (bdi)
KERNEL[1305015566.434478] change /devices/platform/soc-audio/sound/card0 (sound)
UDEV [1305015566.449157] change /devices/platform/soc-audio/sound/card0/controlC0 (sound)
UDEV [1305015566.452270] change /devices/virtual/bdi/default (bdi)
UDEV [1305015566.453613] change /devices/virtual/bdi/7:7 (bdi)
UDEV [1305015566.456268] change /devices/virtual/bdi/mtd-rwmap (bdi)
UDEV [1305015566.457550] change /devices/virtual/bdi/mtd-romap (bdi)
UDEV [1305015566.473754] change /devices/virtual/bdi/mtd-unmap (bdi)
UDEV [1305015567.000915] change /devices/virtual/gpio/gpio157 (gpio)
UDEV [1305015567.109008] change /devices/virtual/gpio/gpiochip0 (gpio)
UDEV [1305015567.115722] change /devices/virtual/gpio/gpiochip128 (gpio)
UDEV [1305015567.205718] change /devices/virtual/gpio/gpiochip160 (gpio)
UDEV [1305015567.313964] change /devices/virtual/gpio/gpiochip32 (gpio)
UDEV [1305015567.316070] change /devices/virtual/gpio/gpiochip64 (gpio)
UDEV [1305015567.318359] change /devices/virtual/gpio/gpiochip96 (gpio)
UDEV [1305015567.328979] change /devices/virtual/graphics/fbcon (graphics)
UDEV [1305015567.429321] change /devices/virtual/mem/full (mem)
UDEV [1305015567.446563] change /devices/virtual/mem/kmsg (mem)
UDEV [1305015567.453918] change /devices/virtual/mem/kmem (mem)
UDEV [1305015567.457031] change /devices/virtual/mem/mem (mem)
UDEV [1305015567.463287] change /devices/virtual/mem/null (mem)
UDEV [1305015567.705779] change /devices/virtual/mem/random (mem)
UDEV [1305015567.718017] change /devices/virtual/mem/urandom (mem)
UDEV [1305015567.744537] change /devices/virtual/mem/zero (mem)
UDEV [1305015567.753051] change /devices/virtual/input/mice (input)
UDEV [1305015567.772094] change /devices/virtual/misc/autofs (misc)
UDEV [1305015567.775390] change /devices/virtual/misc/cpu_dma_latency (misc)
UDEV [1305015567.778228] change /devices/virtual/misc/network_latency (misc)
UDEV [1305015567.840789] change /devices/virtual/misc/network_throughput (misc)
UDEV [1305015567.843536] change /devices/virtual/misc/psaux (misc)
UDEV [1305015567.847473] change /devices/virtual/misc/twl4030-madc (misc)
UDEV [1305015567.849609] change /devices/virtual/misc/ubi_ctrl (misc)
UDEV [1305015567.887817] change /devices/virtual/misc/uinput (misc)
UDEV [1305015567.922485] change /devices/virtual/mtd/mtd0 (mtd)
UDEV [1305015567.937896] change /devices/virtual/mtd/mtd0ro (mtd)
UDEV [1305015568.003173] change /devices/virtual/mtd/mtd1 (mtd)
UDEV [1305015568.012084] change /devices/virtual/mtd/mtd0/mtdblock0 (block)
UDEV [1305015568.015472] change /devices/virtual/mtd/mtd1/mtdblock1 (block)
UDEV [1305015568.026580] change /devices/virtual/mtd/mtd1ro (mtd)
UDEV [1305015568.038726] change /devices/virtual/mtd/mtd2 (mtd)
UDEV [1305015568.039703] change /devices/virtual/mtd/mtd2ro (mtd)
UDEV [1305015568.048431] change /devices/virtual/mtd/mtd3 (mtd)
UDEV [1305015568.052185] change /devices/virtual/mtd/mtd3ro (mtd)
UDEV [1305015568.099761] change /devices/virtual/mtd/mtd4 (mtd)
UDEV [1305015568.106903] change /devices/virtual/mtd/mtd2/mtdblock2 (block)
UDEV [1305015568.109283] change /devices/virtual/mtd/mtd3/mtdblock3 (block)
UDEV [1305015568.112853] change /devices/virtual/mtd/mtd4/mtdblock4 (block)
UDEV [1305015568.121307] change /devices/virtual/mtd/mtd4ro (mtd)
UDEV [1305015568.124603] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa/block/mmcblk0 (block)
UDEV [1305015568.137115] change /devices/virtual/net/lo (net)
UDEV [1305015568.178131] change /devices/virtual/regulator/regulator.0 (regulator)
UDEV [1305015568.182006] change /devices/virtual/sound/timer (sound)
UDEV [1305015568.277160] change /devices/virtual/tty/console (tty)
UDEV [1305015568.314544] change /devices/virtual/tty/ptmx (tty)
UDEV [1305015568.318237] change /devices/virtual/tty/tty (tty)
UDEV [1305015568.343475] change /devices/virtual/tty/tty0 (tty)
UDEV [1305015568.388061] change /devices/virtual/tty/tty10 (tty)
UDEV [1305015568.401367] change /devices/virtual/tty/tty11 (tty)
UDEV [1305015568.418395] change /devices/virtual/tty/tty1 (tty)
UDEV [1305015568.440856] change /devices/virtual/tty/tty12 (tty)
UDEV [1305015568.446563] change /devices/virtual/tty/tty13 (tty)
UDEV [1305015568.456817] change /devices/virtual/tty/tty14 (tty)
UDEV [1305015568.514495] change /devices/virtual/tty/tty16 (tty)
UDEV [1305015568.541534] change /devices/virtual/tty/tty15 (tty)
UDEV [1305015568.553283] change /devices/virtual/tty/tty17 (tty)
UDEV [1305015568.561492] change /devices/virtual/tty/tty18 (tty)
UDEV [1305015568.585998] change /devices/virtual/tty/tty19 (tty)
UDEV [1305015568.688262] change /devices/virtual/tty/tty20 (tty)
UDEV [1305015568.691009] change /devices/virtual/tty/tty2 (tty)
UDEV [1305015568.700622] change /devices/virtual/tty/tty21 (tty)
UDEV [1305015568.711181] change /devices/virtual/tty/tty22 (tty)
UDEV [1305015568.714599] change /devices/virtual/tty/tty23 (tty)
UDEV [1305015568.730621] change /devices/virtual/tty/tty24 (tty)
UDEV [1305015568.756256] change /devices/virtual/tty/tty25 (tty)
UDEV [1305015568.767303] change /devices/virtual/tty/tty26 (tty)
UDEV [1305015568.772643] change /devices/virtual/tty/tty27 (tty)
UDEV [1305015568.789977] change /devices/virtual/tty/tty28 (tty)
UDEV [1305015568.793121] change /devices/virtual/tty/tty29 (tty)
UDEV [1305015568.798400] change /devices/virtual/tty/tty30 (tty)
UDEV [1305015568.811614] change /devices/virtual/tty/tty3 (tty)
UDEV [1305015568.836181] change /devices/virtual/tty/tty31 (tty)
UDEV [1305015568.841735] change /devices/virtual/tty/tty33 (tty)
UDEV [1305015568.860107] change /devices/virtual/tty/tty32 (tty)
UDEV [1305015568.932098] change /devices/virtual/tty/tty34 (tty)
UDEV [1305015568.973815] change /devices/virtual/tty/tty36 (tty)
UDEV [1305015568.974487] change /devices/virtual/tty/tty35 (tty)
UDEV [1305015568.978027] change /devices/virtual/tty/tty37 (tty)
UDEV [1305015568.994140] change /devices/virtual/tty/tty38 (tty)
UDEV [1305015568.997589] change /devices/virtual/tty/tty39 (tty)
UDEV [1305015569.056396] change /devices/virtual/tty/tty40 (tty)
UDEV [1305015569.056976] change /devices/virtual/tty/tty4 (tty)
UDEV [1305015569.072998] change /devices/virtual/tty/tty41 (tty)
UDEV [1305015569.085174] change /devices/virtual/tty/tty42 (tty)
UDEV [1305015569.100036] change /devices/virtual/tty/tty43 (tty)
UDEV [1305015569.110473] change /devices/virtual/tty/tty45 (tty)
UDEV [1305015569.113006] change /devices/virtual/tty/tty44 (tty)
UDEV [1305015569.129852] change /devices/virtual/tty/tty46 (tty)
UDEV [1305015569.139709] change /devices/virtual/tty/tty47 (tty)
UDEV [1305015569.149352] change /devices/virtual/tty/tty48 (tty)
UDEV [1305015569.159179] change /devices/virtual/tty/tty49 (tty)
UDEV [1305015569.184265] change /devices/virtual/tty/tty50 (tty)
UDEV [1305015569.185638] change /devices/virtual/tty/tty5 (tty)
UDEV [1305015569.188934] change /devices/virtual/tty/tty51 (tty)
UDEV [1305015569.206054] change /devices/virtual/tty/tty52 (tty)
UDEV [1305015569.218139] change /devices/virtual/tty/tty53 (tty)
UDEV [1305015569.241577] change /devices/virtual/tty/tty55 (tty)
UDEV [1305015569.242492] change /devices/virtual/tty/tty54 (tty)
UDEV [1305015569.252166] change /devices/virtual/tty/tty56 (tty)
UDEV [1305015569.268524] change /devices/virtual/tty/tty57 (tty)
UDEV [1305015569.277984] change /devices/virtual/tty/tty58 (tty)
UDEV [1305015569.281921] change /devices/virtual/tty/tty59 (tty)
UDEV [1305015569.291839] change /devices/virtual/tty/tty6 (tty)
UDEV [1305015569.307800] change /devices/virtual/tty/tty60 (tty)
UDEV [1305015569.328124] change /devices/virtual/tty/tty62 (tty)
UDEV [1305015569.328857] change /devices/virtual/tty/tty61 (tty)
UDEV [1305015569.332061] change /devices/virtual/tty/tty63 (tty)
UDEV [1305015569.358367] change /devices/virtual/tty/tty8 (tty)
UDEV [1305015569.365356] change /devices/virtual/tty/tty7 (tty)
UDEV [1305015569.369354] change /devices/virtual/tty/tty9 (tty)
UDEV [1305015569.387969] change /devices/virtual/tty/ttyprintk (tty)
UDEV [1305015569.403564] change /devices/virtual/usbmon/usbmon0 (usbmon)
UDEV [1305015569.408294] change /devices/virtual/vc/vcs (vc)
UDEV [1305015569.423797] change /devices/virtual/vc/vcs1 (vc)
UDEV [1305015569.428070] change /devices/virtual/vc/vcs3 (vc)
UDEV [1305015569.442687] change /devices/virtual/vc/vcs2 (vc)
UDEV [1305015569.446350] change /devices/virtual/vc/vcs4 (vc)
UDEV [1305015569.462158] change /devices/virtual/vc/vcs5 (vc)
UDEV [1305015569.465850] change /devices/virtual/vc/vcsa (vc)
UDEV [1305015569.480987] change /devices/virtual/vc/vcs6 (vc)
UDEV [1305015569.484313] change /devices/virtual/vc/vcsa1 (vc)
UDEV [1305015569.509094] change /devices/virtual/vc/vcsa3 (vc)
UDEV [1305015569.516448] change /devices/virtual/vc/vcsa2 (vc)
UDEV [1305015569.519592] change /devices/virtual/vc/vcsa4 (vc)
UDEV [1305015569.536285] change /devices/virtual/vc/vcsa6 (vc)
UDEV [1305015569.538879] change /devices/virtual/vc/vcsa5 (vc)
UDEV [1305015569.554046] change /devices/virtual/vtconsole/vtcon1 (vtconsole)
UDEV [1305015569.555938] change /devices/virtual/vtconsole/vtcon0 (vtconsole)
UDEV [1305015569.638732] change /devices/platform/soc-audio/sound/card0 (sound)
UDEV [1305015571.516021] change /devices/virtual/block/loop4 (block)
UDEV [1305015572.191192] change /devices/virtual/block/ram15 (block)
UDEV [1305015572.193511] change /devices/virtual/block/loop3 (block)
UDEV [1305015572.195739] change /devices/virtual/block/ram2 (block)
UDEV [1305015572.198089] change /devices/virtual/block/ram0 (block)
UDEV [1305015572.200164] change /devices/virtual/block/ram3 (block)
UDEV [1305015572.212982] change /devices/virtual/block/ram14 (block)
UDEV [1305015572.216064] change /devices/virtual/block/loop6 (block)
UDEV [1305015572.218566] change /devices/virtual/block/ram13 (block)
UDEV [1305015572.221069] change /devices/virtual/block/loop7 (block)
UDEV [1305015572.223388] change /devices/virtual/block/loop0 (block)
UDEV [1305015572.227142] change /devices/virtual/block/ram6 (block)
UDEV [1305015572.231201] change /devices/virtual/block/ram11 (block)
UDEV [1305015572.234008] change /devices/virtual/block/ram1 (block)
UDEV [1305015572.237121] change /devices/virtual/block/ram8 (block)
UDEV [1305015572.239776] change /devices/virtual/block/ram9 (block)
UDEV [1305015572.242797] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa/block/mmcblk0/mmcblk0p2 (block)
UDEV [1305015572.246795] change /devices/virtual/block/ram4 (block)
UDEV [1305015572.249755] change /devices/virtual/block/ram10 (block)
UDEV [1305015572.252532] change /devices/virtual/block/ram5 (block)
UDEV [1305015572.255401] change /devices/virtual/block/loop5 (block)
UDEV [1305015572.258727] change /devices/virtual/block/ram12 (block)
UDEV [1305015572.261657] change /devices/virtual/block/ram7 (block)
UDEV [1305015572.264190] change /devices/virtual/block/loop2 (block)
UDEV [1305015572.267028] change /devices/platform/mmci-omap-hs.0/mmc_host/mmc0/mmc0:aaaa/block/mmcblk0/mmcblk0p1 (block)
UDEV [1305015572.270141] change /devices/virtual/block/loop1 (block)

regards,

Koen

Post by Kay Sievers
during the trigger command above will be useful too.
Kay

Kay Sievers

2011-05-10 08:36:28 UTC

time (udevadm trigger; udevadm settle)

real 3m0.475s

3 Minutes?

UDEV [1305015564.609832] change /devices/omapdss/display0 (omapdss)

...

UDEV [1305015572.270141] change /devices/virtual/block/loop1 (block)

Between the first and last event are 7 seconds. Where are the 3
minutes spent? What do I miss?

Kay

Harald Hoyer

2011-05-10 08:40:03 UTC

Post by Kay Sievers
time (udevadm trigger; udevadm settle)

real 3m0.475s

3 Minutes?

Post by Koen Kooi
UDEV [1305015564.609832] change /devices/omapdss/display0 (omapdss)

...

Post by Koen Kooi
UDEV [1305015572.270141] change /devices/virtual/block/loop1 (block)

Between the first and last event are 7 seconds. Where are the 3
minutes spent? What do I miss?
Kay

monitor does not show, when a event has finished/settled

Harald Hoyer

2011-05-10 09:02:35 UTC

Post by Kay Sievers
time (udevadm trigger; udevadm settle)

real 3m0.475s

3 Minutes?

UDEV [1305015564.609832] change /devices/omapdss/display0 (omapdss)

...

UDEV [1305015572.270141] change /devices/virtual/block/loop1 (block)

Between the first and last event are 7 seconds. Where are the 3
minutes spent? What do I miss?
Kay

monitor does not show, when a event has finished/settled

I stand corrected... it does.

Koen Kooi

2011-05-10 09:26:00 UTC

Post by Kay Sievers
time (udevadm trigger; udevadm settle)

real 3m0.475s

3 Minutes?

It's 3 minutes exactly (+trigger and shell overhead) every time, so I guess I'm hitting some kind of timeout

Post by Koen Kooi
UDEV [1305015564.609832] change /devices/omapdss/display0 (omapdss)

...

Post by Koen Kooi
UDEV [1305015572.270141] change /devices/virtual/block/loop1 (block)

Between the first and last event are 7 seconds. Where are the 3
minutes spent? What do I miss?
Kay

Kay Sievers

2011-05-10 09:55:19 UTC

time (udevadm trigger; udevadm settle)

real 3m0.475s

3 Minutes?

It's 3 minutes exactly (+trigger and shell overhead) every time, so I guess I'm hitting some kind of timeout

It's 180 sec, yeah. Doesn't it print what device it waited for?

If you add: "... settle --timeout=10" does it print something?

Kay

Koen Kooi

2011-05-10 10:22:43 UTC

Post by Kay Sievers
time (udevadm trigger; udevadm settle)

real 3m0.475s

3 Minutes?

It's 3 minutes exactly (+trigger and shell overhead) every time, so I guess I'm hitting some kind of timeout

It's 180 sec, yeah. Doesn't it print what device it waited for?

udevadm doesn't

Post by Kay Sievers
If you add: "... settle --timeout=10" does it print something?

***@beagleboard-systemd:~# time ( udevadm trigger ; udevadm settle --timeout=10)

real 0m10.427s
user 0m0.023s
sys 0m0.102s

Changing it to '1' doesn't print anything either. Should udev-monitor print the device it's waiting for?

regards,

Koen

Kay Sievers

2011-05-10 10:28:51 UTC

Post by Koen Kooi
It's 3 minutes exactly (+trigger and shell overhead) every time, so I guess I'm hitting some kind of timeout

It's 180 sec, yeah. Doesn't it print what device it waited for?

udevadm doesn't

Post by Kay Sievers
If you add: "... settle --timeout=10" does it print something?

real 0m10.427s
user 0m0.023s
sys 0m0.102s
Changing it to '1' doesn't print anything either. Should udev-monitor print the device it's waiting for?

Hmm, settle should print it like (I added sleep 15 to /dev/null):

time (udevadm trigger; udevadm settle --timeout=10)

udevadm settle - timeout of 10 seconds reached, the event queue contains:
/sys/devices/virtual/mem/null (5709)

real 0m10.131s

What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

Kay

Koen Kooi

2011-05-10 11:01:55 UTC

Post by Koen Kooi
It's 3 minutes exactly (+trigger and shell overhead) every time, so I guess I'm hitting some kind of timeout

It's 180 sec, yeah. Doesn't it print what device it waited for?

udevadm doesn't

Post by Kay Sievers
If you add: "... settle --timeout=10" does it print something?

real 0m10.427s
user 0m0.023s
sys 0m0.102s
Changing it to '1' doesn't print anything either. Should udev-monitor print the device it's waiting for?

time (udevadm trigger; udevadm settle --timeout=10)
/sys/devices/virtual/mem/null (5709)
real 0m10.131s
What udev version is that?

168, I'll try git in a moment.

Koen Kooi

2011-05-10 11:47:21 UTC

Post by Koen Kooi
It's 3 minutes exactly (+trigger and shell overhead) every time, so I guess I'm hitting some kind of timeout

It's 180 sec, yeah. Doesn't it print what device it waited for?

udevadm doesn't

Post by Kay Sievers
If you add: "... settle --timeout=10" does it print something?

real 0m10.427s
user 0m0.023s
sys 0m0.102s
Changing it to '1' doesn't print anything either. Should udev-monitor print the device it's waiting for?

time (udevadm trigger; udevadm settle --timeout=10)
/sys/devices/virtual/mem/null (5709)
real 0m10.131s
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967] <27>udevd[88]: unable to receive ctrl connection: Function not implemented

Time to rebuild eglibc I guess.

Lennart Poettering

2011-05-10 12:09:25 UTC

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967] <27>udevd[88]: unable to receive ctrl connection: Function not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?

Lennart

--
Lennart Poettering - Red Hat, Inc.

Kay Sievers

2011-05-10 12:11:07 UTC

On Tue, May 10, 2011 at 14:09, Lennart Poettering

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967] <27>udevd[88]: unable to receive ctrl connection: Function not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?

accept4() it is.

Kay

Harald Hoyer

2011-05-10 12:11:59 UTC

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967]<27>udevd[88]: unable to receive ctrl connection: Function not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?
Lennart

Might be time, for a .config list of required features :-)

Kay Sievers

2011-05-10 12:14:37 UTC

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967]<27>udevd[88]: unable to receive ctrl connection: Function
not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?
Lennart

Might be time, for a .config list of required features :-)

It should be in INSTALL.

Kay

Lennart Poettering

2011-05-10 12:14:53 UTC

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967]<27>udevd[88]: unable to receive ctrl connection: Function not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?
Lennart

Might be time, for a .config list of required features :-)

Neither accept4 nor AF_UNIX/SOCK_SEQPACKET are optional kernel
features. However, they are relatively new kernel features and some
archs had trouble keeping up in enabling them even though the code
exists in the kernel.

Currently our README mentions that we need kernel >= 2.6.30. It might
make sense to include another line mentioning that on ARM/MIPS/others
you need an even newer one.

Lennart

--
Lennart Poettering - Red Hat, Inc.

Koen Kooi

2011-05-10 12:19:24 UTC

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967]<27>udevd[88]: unable to receive ctrl connection: Function not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?
Lennart

Might be time, for a .config list of required features :-)

Neither accept4 nor AF_UNIX/SOCK_SEQPACKET are optional kernel
features. However, they are relatively new kernel features and some
archs had trouble keeping up in enabling them even though the code
exists in the kernel.
Currently our README mentions that we need kernel >= 2.6.30. It might
make sense to include another line mentioning that on ARM/MIPS/others
you need an even newer one.

Udev has this: http://git.kernel.org/?p=linux/hotplug/udev.git;a=patch;h=67a77c8bf299f6264f001677becd056316ebce2f

I backported that kernel patch to 2.6.32 since I can test that, but the systemd tests were all run with 2.6.37. The problem is that eglibc was built against yet another different version of headers, which lacked the syscall for arm.

regards,

Koen

Koen Kooi

2011-05-10 15:34:10 UTC

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967]<27>udevd[88]: unable to receive ctrl connection: Function not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?
Lennart

Might be time, for a .config list of required features :-)

Neither accept4 nor AF_UNIX/SOCK_SEQPACKET are optional kernel
features. However, they are relatively new kernel features and some
archs had trouble keeping up in enabling them even though the code
exists in the kernel.
Currently our README mentions that we need kernel >= 2.6.30. It might
make sense to include another line mentioning that on ARM/MIPS/others
you need an even newer one.

So after rebuilding eglibc against 2.6.37 headers and udev from git:

***@beagleboard-systemd:~# time ( udevadm trigger ; udevadm settle )

real 0m10.475s
user 0m0.000s
sys 0m0.133s

And the matching systemd-analyze plot: http://dominion.thruhere.net/koen/angstrom/systemd/git-with-accept4.svg

I'll do some more digging after linuxtag.

regards,

Koen

Lennart Poettering

2011-05-10 15:39:01 UTC

Post by Kay Sievers
What udev version is that? I'm running current -git, maybe something
was broken, don't know it though.

[ 122.891967]<27>udevd[88]: unable to receive ctrl connection: Function not implemented
Time to rebuild eglibc I guess.

Hmm, does your kernel lack AF_UNIX/SOCK_SEQPACKET support?
Lennart

Might be time, for a .config list of required features :-)

Neither accept4 nor AF_UNIX/SOCK_SEQPACKET are optional kernel
features. However, they are relatively new kernel features and some
archs had trouble keeping up in enabling them even though the code
exists in the kernel.
Currently our README mentions that we need kernel >= 2.6.30. It might
make sense to include another line mentioning that on ARM/MIPS/others
you need an even newer one.

real 0m10.475s
user 0m0.000s
sys 0m0.133s
And the matching systemd-analyze plot: http://dominion.thruhere.net/koen/angstrom/systemd/git-with-accept4.svg

Seems the triggering takes a about a second. Which is not particularly
good, but I guess not the most pressing issue.

It seems the devices are probed very slowly, and even the tty devices
take ages to complete. i.e. between most VC tts to showing up almost a
second passes.

Might be useful to run bootchart on this to get a more finegrained idea
what udev is doing there.

Lennart

--
Lennart Poettering - Red Hat, Inc.

Kay Sievers

2011-05-10 17:01:32 UTC

real 0m10.475s

Ok, at least the timeout issue seems gone.

How many device do you have?
find /sys -name uevent | wc -l

How many devices have a modalias:
find /sys -name modalias | wc -l

And the matching systemd-analyze plot: http://dominion.thruhere.net/koen/angstrom/systemd/git-with-accept4.svg

What does:
time udevadm test /sys/class/tty/ttyO0
show?

If you run it only for certain subsystems, what does it give you:
time ( udevadm trigger -s block; udevadm settle )
time ( udevadm trigger -s tty; udevadm settle )

I'll do some more digging after linuxtag.

I'll be around Fri+Sat. Maybe we'll walk into each other ...

Kay

Koen Kooi

2011-05-10 18:37:23 UTC

Post by Koen Kooi
real 0m10.475s

Ok, at least the timeout issue seems gone.
How many device do you have?
find /sys -name uevent | wc -l

***@beagleboard-systemd:/usr/bin# find /sys -name uevent | wc -l
462

Post by Kay Sievers
find /sys -name modalias | wc -l

***@beagleboard-systemd:/usr/bin# find /sys -name modalias | wc -l
84

Post by Koen Kooi
And the matching systemd-analyze plot: http://dominion.thruhere.net/koen/angstrom/systemd/git-with-accept4.svg

time udevadm test /sys/class/tty/ttyO0
show?

***@beagleboard-systemd:/usr/bin# time udevadm test /sys/class/tty/ttyO0
run_command: calling: test
udevadm_test: version 168
This program is for debugging only, it does not run any program,
specified by a RUN key. It may show incorrect results, because
some values may be different, or not available at a simulation run.

parse_file: reading '/lib/udev/rules.d/42-qemu-usb.rules' as rules file
parse_file: reading '/lib/udev/rules.d/50-firmware.rules' as rules file
parse_file: reading '/etc/udev/rules.d/50-udev-default.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-cdrom_id.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-floppy.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-persistent-alsa.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-persistent-input.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-persistent-serial.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-persistent-storage-tape.rules' as rules file
parse_file: reading '/etc/udev/rules.d/60-persistent-storage.rules' as rules file
parse_file: reading '/lib/udev/rules.d/60-persistent-v4l.rules' as rules file
parse_file: reading '/lib/udev/rules.d/61-mobile-action.rules' as rules file
parse_file: reading '/lib/udev/rules.d/61-persistent-storage-edd.rules' as rules file
parse_file: reading '/lib/udev/rules.d/70-acl.rules' as rules file
parse_file: reading '/lib/udev/rules.d/70-hid2hci.rules' as rules file
parse_file: reading '/lib/udev/rules.d/75-cd-aliases-generator.rules' as rules file
parse_file: reading '/etc/udev/rules.d/75-net-description.rules' as rules file
parse_file: reading '/lib/udev/rules.d/75-persistent-net-generator.rules' as rules file
parse_file: reading '/lib/udev/rules.d/75-probe_mtd.rules' as rules file
parse_file: reading '/etc/udev/rules.d/75-tty-description.rules' as rules file
parse_file: reading '/etc/udev/rules.d/78-sound-card.rules' as rules file
parse_file: reading '/etc/udev/rules.d/80-drivers.rules' as rules file
parse_file: reading '/lib/udev/rules.d/95-keyboard-force-release.rules' as rules file
parse_file: reading '/lib/udev/rules.d/95-keymap.rules' as rules file
parse_file: reading '/etc/udev/rules.d/95-udev-late.rules' as rules file
parse_file: reading '/lib/udev/rules.d/97-bluetooth-hid2hci.rules' as rules file
parse_file: reading '/lib/udev/rules.d/97-bluetooth.rules' as rules file
parse_file: reading '/lib/udev/rules.d/99-systemd.rules' as rules file
parse_file: reading '/etc/udev/rules.d/local.rules' as rules file
parse_file: reading '/etc/udev/rules.d/permissions.rules' as rules file
parse_file: reading '/etc/udev/rules.d/run.rules' as rules file
parse_file: reading '/etc/udev/rules.d/udev.rules' as rules file
udev_rules_new: rules use 27348 bytes tokens (2279 * 12 bytes), 15978 bytes buffer
udev_rules_new: temporary index used 16860 bytes (843 * 20 bytes)
udev_device_new_from_syspath: device 0x1fd50d0 has devpath '/devices/platform/omap/omap_uart.0/tty/ttyO0'
udev_device_new_from_syspath: device 0x1fe48e0 has devpath '/devices/platform/omap/omap_uart.0/tty/ttyO0'
udev_device_read_db: device 0x1fe48e0 filled with db file data
udev_rules_apply_to_event: GROUP 20 /etc/udev/rules.d/50-udev-default.rules:11
udev_device_new_from_syspath: device 0x1fdf078 has devpath '/devices/platform/omap/omap_uart.0'
udev_device_new_from_syspath: device 0x1fdf2b8 has devpath '/devices/platform/omap'
udev_device_new_from_syspath: device 0x1fdf438 has devpath '/devices/platform'
udev_rules_apply_to_event: GROUP 20 /etc/udev/rules.d/permissions.rules:43
udev_rules_apply_to_event: RUN 'socket:/org/kernel/udev/monitor' /etc/udev/rules.d/run.rules:2
udev_event_execute_rules: no node name set, will use kernel supplied name 'ttyO0'
udev_node_add: creating device node '/dev/ttyO0', devnum=252:0, mode=0660, uid=0, gid=20
udev_node_mknod: preserve file '/dev/ttyO0', because it has correct dev_t
udev_node_mknod: preserve permissions /dev/ttyO0, 020660, uid=0, gid=20
node_symlink: preserve already existing symlink '/dev/char/252:0' to '../ttyO0'
udev_device_update_db: created db file '/run/udev/data/c252:0' for '/devices/platform/omap/omap_uart.0/tty/ttyO0'
UDEV_LOG=6
DEVPATH=/devices/platform/omap/omap_uart.0/tty/ttyO0
MAJOR=252
MINOR=0
DEVNAME=/dev/ttyO0
ACTION=add
SUBSYSTEM=tty
TAGS=:systemd:
run: 'socket:/org/kernel/udev/monitor'

real 0m0.065s
user 0m0.023s
sys 0m0.016s

Post by Kay Sievers
time ( udevadm trigger -s block; udevadm settle )

That triggers a preempt bug and oopses the kernel.

Post by Kay Sievers
time ( udevadm trigger -s tty; udevadm settle )

***@beagleboard-systemd:~# time ( udevadm trigger -s tty; udevadm settle )

real 0m1.203s
user 0m0.000s
sys 0m0.031s

Post by Koen Kooi
I'll do some more digging after linuxtag.

I'll be around Fri+Sat. Maybe we'll walk into each other ...

I'll attend the systemd talk :)

regards,

Koen

Post by Kay Sievers
Kay

Kay Sievers

2011-05-10 20:33:37 UTC

real 0m10.475s

Ok, at least the timeout issue seems gone.
How many device do you have?
find /sys -name uevent | wc -l

462

Post by Kay Sievers
find /sys -name modalias | wc -l

That looks all fine.

Post by Koen Kooi
parse_file: reading '/etc/udev/rules.d/50-udev-default.rules' as rules file
parse_file: reading '/etc/udev/rules.d/95-udev-late.rules' as rules file
parse_file: reading '/etc/udev/rules.d/local.rules' as rules file
parse_file: reading '/etc/udev/rules.d/permissions.rules' as rules file
parse_file: reading '/etc/udev/rules.d/run.rules' as rules file
parse_file: reading '/etc/udev/rules.d/udev.rules' as rules file

What are these files? There is no default udev rules file in /etc.
'late' is years old, and the ones without number prefix look like they
come from something else, and they shouldn't be there.

Post by Koen Kooi
real 0m0.065s

Post by Kay Sievers
time ( udevadm trigger -s block; udevadm settle )

That triggers a preempt bug and oopses the kernel.

Great! :)

Post by Kay Sievers
time ( udevadm trigger -s tty; udevadm settle )

real 0m1.203s
user 0m0.000s
sys 0m0.031s

That's seems very long compared to the above run for a single tty.

Post by Kay Sievers
I'll be around Fri+Sat. Maybe we'll walk into each other ...

I'll attend the systemd talk :)

Good. See you there.

Kay

microcai

2011-05-10 05:40:02 UTC

Correct.

That's actually dead fast

I had an ARM9 with Gentoo + systemd installed.

Systemd itself boot system in about 20s. But it tooks 10s+ to mount the
yaffs2 root fs. So, 30s+ to boot.

Hope this data will be useful to you.

Post by Koen Kooi
regards,
Koen
_______________________________________________
systemd-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Scott James Remnant

2011-05-12 16:01:04 UTC

Post by Scott James Remnant

On Mon, May 9, 2011 at 12:43 PM, Lennart Poettering

Post by Scott James Remnant
This means there are a large number of devices already known to the

kernel

Post by Scott James Remnant
at the point that systemd starts, especially if you build the drivers

into

Post by Scott James Remnant
the kernel for those devices. It's possible to get going straight away

with

Post by Scott James Remnant
those devices. But relying on udevd tagging them means you end up waiting
around for udevd to start, and worse! since udevd doesn't apply rules to
existing devices on startup, you have to wait around for "udevadm

trigger"

Post by Scott James Remnant
to be run.

That's actually dead fast, and systemd picks up those devices as they
show up. The trigger can hence finish after we already preceeded
booting. In fact, with the new netlink actviation in newest udev and
systemd we can start the trigger at the same time as udev itself.
We cannot really mount file systems before their block devices have
shown up and have been probed, and we exactly wait for that, and no
longer. To do fsck/mount we need udev to have run for that device,
there's no realistic way around that.
It isn't "dead fast" enough.

The problem is that when you run udevadm trigger, it doesn't immediately
spawn 700 processes to handle the events. So you end up with situations
where input devices aren't available to X because the system is too busy
probing for filesystems, or loading sound card drivers.

And we really *can* mount file systems before their block devices have been
probed (they show up as soon as the driver's loaded). Actually the probing
code is exactly the kind of code I would want to delay; filesystems can be
mounted by GPT ids which doesn't require the expensive block device probing.

To my mind, we would want the filesystem mounted as soon as the kernel event
arrives, whether or not udevd is running and whether or not udevadm trigger
has been run.

Post by Scott James Remnant
udev isn't just about symlinks. It's about perms and primarily about
meta data, and a lot of other stuff too.
perms are increasingly irrelevant due to ACLs, which your plans are moving

to systemd, no?

Scott

Lennart Poettering

2011-05-16 17:42:10 UTC