Discussion:
How to implement fsck progress report with systemd and plymouth?
(too old to reply)
f***@gmail.com
2011-04-22 07:04:46 UTC
Permalink
Hi all,

plymouth in Ubuntu 10.04 supports fsck progress report, and also
provides a chance for user to cancel running fsck. How to implement
this feature with systemd and plymouth?

I did some investigation, found:
1. ubuntu patches on_update() of plymouth/src/main.c, it will filter
out status message from fsck, if message starts with "fsck:".
2. ubuntu provides a plymouth theme -- ubuntu-logo, which is of type
script, and has some fsck related stuff.

I guess we need:
1. Support "detailed status report" in plymouth, e.g. colon separated fields.
2. Hook fsck detailed message to each theme in plymouth ?
3. Emit fsck message to plymouth in systemd-fsck.
4. How to support user-cancellable fsck?

Any idea?
--
Regards,
- cee1
Lennart Poettering
2011-04-24 21:19:44 UTC
Permalink
Post by f***@gmail.com
Hi all,
plymouth in Ubuntu 10.04 supports fsck progress report, and also
provides a chance for user to cancel running fsck. How to implement
this feature with systemd and plymouth?
It's actually a really hard problem. For example, what do you do when
you have multiple disks which are fsck'ed in parallel or serial? You'd
then have to integrate their individual progress bars somehow.
Post by f***@gmail.com
1. ubuntu patches on_update() of plymouth/src/main.c, it will filter
out status message from fsck, if message starts with "fsck:".
2. ubuntu provides a plymouth theme -- ubuntu-logo, which is of type
script, and has some fsck related stuff.
1. Support "detailed status report" in plymouth, e.g. colon separated fields.
2. Hook fsck detailed message to each theme in plymouth ?
3. Emit fsck message to plymouth in systemd-fsck.
4. How to support user-cancellable fsck?
Any idea?
Here's what I'd suggest: We already have systemd-fsck, which we could
beef up slightly to pass progress data to a tiny AF_UNIX socket
activated service (let's call it fsckd) which integrates the data from
multiple instances of systemd-fsck and sends it off to plymouth. And if
a cancel request is received from plymouth it would send off SIGINT to
the the systemd-fsck instances which it got data from.

This could be done in a relative easy and elegant way by having
systemd-fsck just pass the socket it created as channel to fsckd as -C
fd to to fsck. That way only fsckd would have to parse the progress data
from fsck, and it would talk directly to fsck which would allow fsckd to
figure out the fsck PID via SCM_CREDENTIALS, so that fsckd would know
what to kill.

Lennart
--
Lennart Poettering - Red Hat, Inc.
f***@gmail.com
2011-04-25 02:10:02 UTC
Permalink
Post by Lennart Poettering
Post by f***@gmail.com
Hi all,
plymouth in Ubuntu 10.04 supports fsck progress report, and also
provides a chance for user to cancel running fsck. How to implement
this feature with systemd and plymouth?
It's actually a really hard problem. For example, what do you do when
you have multiple disks which are fsck'ed in parallel or serial? You'd
then have to integrate their individual progress bars somehow.
It seems ubuntu's implementation uses limited 'progress bars'. I guess
any more fsck progress reports will no be displayed.
Post by Lennart Poettering
Post by f***@gmail.com
1. ubuntu patches on_update() of plymouth/src/main.c, it will filter
out status message from fsck, if message starts with "fsck:".
2. ubuntu provides a plymouth theme -- ubuntu-logo, which is of type
script, and has some fsck related stuff.
1. Support "detailed status report" in plymouth, e.g. colon separated fields.
2. Hook fsck detailed message to each theme in plymouth ?
3. Emit fsck message to plymouth in systemd-fsck.
4. How to support user-cancellable fsck?
Any idea?
Here's what I'd suggest: We already have systemd-fsck, which we could
beef up slightly to pass progress data to a tiny AF_UNIX socket
activated service (let's call it fsckd) which integrates the data from
multiple instances of systemd-fsck and sends it off to plymouth. And if
a cancel request is received from plymouth it would send off SIGINT to
the the systemd-fsck instances which it got data from.
This could be done in a relative easy and elegant way by having
systemd-fsck just pass the socket it created as channel to fsckd as -C
fd to to fsck. That way only fsckd would have to parse the progress data
from fsck, and it would talk directly to fsck which would allow fsckd to
figure out the fsck PID via SCM_CREDENTIALS, so that fsckd would know
what to kill.
Good idea. Thanks.
--
Regards,
- cee1
Ray Strode
2011-04-26 20:59:35 UTC
Permalink
Hi,

On Sun, Apr 24, 2011 at 5:19 PM, Lennart Poettering
Post by Lennart Poettering
Post by f***@gmail.com
plymouth in Ubuntu 10.04 supports fsck progress report, and also
provides a chance for user to cancel running fsck. How to implement
this feature with systemd and plymouth?
It's actually a really hard problem. For example, what do you do when
you have multiple disks which are fsck'ed in parallel or serial? You'd
then have to integrate their individual progress bars somehow.
I guess for a lot of themes, just one progress bar at a time for

"Disks getting checked"

should be enough, but should probably be up to the theme to decide how
detailed to go.
Post by Lennart Poettering
Here's what I'd suggest: We already have systemd-fsck, which we could
beef up slightly to pass progress data to a tiny AF_UNIX socket
activated service (let's call it fsckd) which integrates the data from
multiple instances of systemd-fsck and sends it off to plymouth. And if
a cancel request is received from plymouth it would send off SIGINT to
the the systemd-fsck instances which it got data from.
So on the plymouth side of the fence, I had origionally envisioned
something analagous to how passwords are handled. So instead of

plymouth ask-for-password --command "cryptsetup ...."

something like

plymouth show-filesystem-check --command "fsck ..."

So in your proposal (which sounds fine) we wouldn't have that at all,
but instead of have fsckd take that role.

So we'll need some new protocol, I guess, to notify plymouthd of a new
fsck operations as they come in, and protocol for updating the status
of existing fsck operations. Then plymouthd will just pass that info
on to the theme and let it figure out what to do with it. I guess
fsckd will have multiple concurrent connections to plymouthd that
don't close until the fscks complete. That way we have a
communication channel back to fsckd to report cancelation.

--Ray
Lennart Poettering
2011-04-26 21:11:00 UTC
Permalink
Post by Ray Strode
Post by Lennart Poettering
Here's what I'd suggest: We already have systemd-fsck, which we could
beef up slightly to pass progress data to a tiny AF_UNIX socket
activated service (let's call it fsckd) which integrates the data from
multiple instances of systemd-fsck and sends it off to plymouth. And if
a cancel request is received from plymouth it would send off SIGINT to
the the systemd-fsck instances which it got data from.
So on the plymouth side of the fence, I had origionally envisioned
something analagous to how passwords are handled. So instead of
plymouth ask-for-password --command "cryptsetup ...."
something like
plymouth show-filesystem-check --command "fsck ..."
So in your proposal (which sounds fine) we wouldn't have that at all,
but instead of have fsckd take that role.
So we'll need some new protocol, I guess, to notify plymouthd of a new
fsck operations as they come in, and protocol for updating the status
of existing fsck operations. Then plymouthd will just pass that info
on to the theme and let it figure out what to do with it. I guess
fsckd will have multiple concurrent connections to plymouthd that
don't close until the fscks complete. That way we have a
communication channel back to fsckd to report cancelation.
I would suggest to leave the parsing of the fsck -C output to fsckd, as
well as the integration of multiple of these streams. fsckd would then
only pass one stream of progress bar information to plymouth. The
handling in Plymouth would be very simple: you'd just add a single
request which gets a percentage. You show the bar on the first of those
you get, you hide the bar if the percentage you get is >= 100. And all
the dirty stuff would stay in fsckd. fsckd would probably just take the
minimum of the most recent percentage data from all its clients and pass
that on to plymouth.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Ray Strode
2011-04-26 21:53:42 UTC
Permalink
Hi,

On Tue, Apr 26, 2011 at 5:11 PM, Lennart Poettering
Post by Lennart Poettering
I would suggest to leave the parsing of the fsck -C output to fsckd, as
well as the integration of multiple of these streams. fsckd would then
only pass one stream of progress bar information to plymouth. The
handling in Plymouth would be very simple: you'd just add a single
request which gets a percentage. You show the bar on the first of those
you get, you hide the bar if the percentage you get is >= 100. And all
the dirty stuff would stay in fsckd. fsckd would probably just take the
minimum of the most recent percentage data from all its clients and pass
that on to plymouth.
Sounds fine.

--Ray
f***@gmail.com
2011-04-27 09:04:16 UTC
Permalink
Hi,
Post by Lennart Poettering
I would suggest to leave the parsing of the fsck -C output to fsckd, as
well as the integration of multiple of these streams. fsckd would then
only pass one stream of progress bar information to plymouth. The
handling in Plymouth would be very simple: you'd just add a single
request which gets a percentage. You show the bar on the first of those
you get, you hide the bar if the percentage you get is >= 100. And all
the dirty stuff would stay in fsckd. fsckd would probably just take the
minimum of the most recent percentage data from all its clients and pass
that on to plymouth.
I'm currently trying to do similar, but:
1. Parsing "fsck -C" output in systemd-fsck. All instances of
systemd-fsck send this progress status to plymouth, and let theme
decide how to display them on screen.
2. If user press e.g. 'C' to cancel a certain fsck job, then plymouth
will send SIGINT to corresponding instance of systemd-fsck.

This saves fsckd in "plymouth <-> fsckd <-> n * systemd-fsck".
--
Regards,
- cee1
f***@gmail.com
2011-06-20 06:49:53 UTC
Permalink
Hi all,
Hi,
Post by Lennart Poettering
I would suggest to leave the parsing of the fsck -C output to fsckd, as
well as the integration of multiple of these streams. fsckd would then
only pass one stream of progress bar information to plymouth. The
handling in Plymouth would be very simple: you'd just add a single
request which gets a percentage. You show the bar on the first of those
you get, you hide the bar if the percentage you get is >= 100. And all
the dirty stuff would stay in fsckd. fsckd would probably just take the
minimum of the most recent percentage data from all its clients and pass
that on to plymouth.
Lennart, you're right, this is the minimal and nature way to implement
the feature. Though I didn't feel very well to invent a fsckd which
was just a bridge between systemd-fsck and plymouth.

Here was my implementation: (The patches are in the attachment, I
guess it may not be merged, hence didn't split the patches, sorry)
1. systemd-fsck will read progress status from fsck, and send it to
plymouth periodicity.
2. plymouth dispatches progress status to
plugins/splash/*/plugin.c(currently only two-step plugin support it),
and let it display the progress status and response for user's cancel
requests.

Note:
1. If more than one systemd-fsck instances running meanwhile, plymouth
will display the first N progress status(and append a "...", if num of
systemd-fsck instances > N).
2. A new protocol is added to plymouth to transport progress status info:
"U\003" + "status" + "detailed_status"
* "status": nr_of_bytes + status string('\0' terminated). The status
string is something like "fsck:/dev/sda1"
* "detailed_statis": nr_of_bytes + detailed status string('\0'
terminated). The detailed status string is the progress status info.

Test:
1. Make sure the following options are compiled into kernel:
"""
Device Drivers ->
{*} Connector - unified userspace <-> kernelspace linker -->
[*] Report process events to userspace


Networking options -> netfilter
"""
2. Set plymouth's default theme to spinner.
3. touch /forcefsck
4. reboot


Change summary:
= systemd =
== util.c[h] ==
1. alarm_ns(), similar to alarm() but can choose clock_id, and is ns precision.
2. sane_dup2(), similar to dup2() but retry for EINTR.
3. read_all().
4. spawn_async_with_pipes().

== utf8-util.c[h] ==
New files utf8-util.h and utf8-util.c, code mainly ported from glib
to add utf8 manipulation functions:
1. utf8_find_prev_char()
2. utf8_validate()
3. convert_with_iconv()
4. get_charset()
5. locale_to_utf8()
6. utf8_merge_backspace_char(), removes "bubbles" like "/\b-\b" from a
utf8 line.

== fsck.c ==
if defined FSCK_USE_PRETTY_PROGRESS_INFO, systemd-fsck will use
progress status info formatted by fsck (e.g. "home|=====> 30.6%")

Note, this macro was added because not all fsck.* support -C option.

= plymouth =
== ply-boot-splash-plugin.c[h] ==
1. Add (*update_status_detailed)(...) for ply_boot_splash_plugin_interface_t
2. Add (*on_key_stroke)(...) for ply_boot_splash_plugin_interface_t

== ply-boot-splash.c[h] ==
1. update_status_detailed() was added to process the new protocol
2. ply_boot_splash_on_key_stroke(), let splash plugin have a change to
response user's input.

== main.c ==
1. on_update() hooks to update_status_detailed()
2. on_keyboard_input() hooks to ply_boot_splash_on_key_stroke()

== ply-boot-server.c[h] ==
1. ply_boot_connection_read_request(), add support to the new
protocol(prototype changed!)
2. ply_boot_connection_on_request(), add support to the new protocol
3. (*ply_boot_server_update_handler_t)(...), add support to the new
protocol(prototype changed!);

== two-step/plugin.c ==
Add support to display detailed status

== themes/glow/glow.plymouth.in, themes/spinner/spinner.plymouth.in ==
"""
[fsck]
FsckPrompt=Checking filesystem:
FsckPrompt[zh_CN]=检查文件系统:
CancelFsckPrompt=Press 'C' to skip filesystem check
CancelFsckPrompt[zh_CN]=按䞋倧写'C'键来跳过文件系统检查
CancelKey=C
"""

== ply-key-file.h ==
1. ply_key_file_get_value_locale(), similar to g_key_file_get_locale_string()

== ply-utils.h ==
1. ply_strchug(), ply_strchomp(), ply_strstrip(),
ply_get_locale_variants(), ply_get_language_names() port from glib,
used by ply_key_file_get_value_locale()
2. Move ply_get_process_command_line() and
ply_get_process_parent_pid() to ply-proc.h

== ply-proc.c[h](New files) ==
1. ply_get_process_command_line() and ply_get_process_parent_pid():
moved from ply-utils.h
2. ply_proc_exit_notifier_get(), ply_proc_exit_notifier_put(),
ply_proc_exit_notifier_reset()
ply_proc_exit_notifier_is_attched_event_loop(),
ply_proc_exit_notifier_attach_event_loop(),
ply_proc_exit_notifier_add_exit_cb()
Add a process exit notifier, based on cn_proc[1], it was reference counting.
Basic usage:
1. ply_proc_exit_notifier_get()
2. if ply_proc_exit_notifier_is_attched_event_loop()
ply_proc_exit_notifier_attach_event_loop()
3. ply_proc_exit_notifier_add_exit_cb() calls cb if has process exited.
4. ply_proc_exit_notifier_put()

ply_proc_exit_notifier_reset() will always clear reference count and
release related resources.


Regards,
-- cee1
----
[1] http://netsplit.com/2011/02/09/the-proc-connector-and-socket-filters/
f***@gmail.com
2011-06-29 09:26:20 UTC
Permalink
Post by f***@gmail.com
Hi all,
1. systemd-fsck will read progress status from fsck, and send it to
plymouth periodicity.
For those who are interested, this is the second version of
"systemd-Add-fsck-plymouth-bridge.patch".
Changes since V1:
1. Fix skipped fsck-root. Now strdup device before udev_device_unref,
thus fsck will not quit early with bad device name.
2. Fix systemd-fsck abortion if "splash" doesn't appear in /proc/cmdline.
--
Regards,
- cee1
Continue reading on narkive:
Loading...