Discussion:
Documentation question
(too old to reply)
David Lambert
2012-02-28 20:51:01 UTC
Permalink
Please excuse if this is trivial, but I am a systemd newbie.
I am running systemd on Angstrom/Beaglebone. After a couple of days I
notice that the process systemd-journald had grown and was ultimately
killed by the kernel's OOM handler. I investigated the
systemd-journald.conf file which stated:

# See system-journald.conf(5) for details

I presume that this is a reference to a man page, but I have downloaded
systemd-43 and cannot find this documentation. I would appreciate any
links to this documentation and/or any suggestions as to where the
problem nay lie.

Best,

Dave.
David Lambert
2012-02-29 03:19:20 UTC
Permalink
I have got a little further in diagnosing this "memory leak" of
systemd-journal. It appears to worsen when I have a process that emits
data to stdout. I presume that systemd-journal is attempting to put this
somewhere but failing, and consequently is buffering it in its own
process. Where is this logging supposed to go, and can I redirect it to
somewhere benign such as /dev/null?

Dave.
Post by David Lambert
Please excuse if this is trivial, but I am a systemd newbie.
I am running systemd on Angstrom/Beaglebone. After a couple of days I
notice that the process systemd-journald had grown and was ultimately
killed by the kernel's OOM handler. I investigated the
# See system-journald.conf(5) for details
I presume that this is a reference to a man page, but I have
downloaded systemd-43 and cannot find this documentation. I would
appreciate any links to this documentation and/or any suggestions as
to where the problem nay lie.
Best,
Dave.
_______________________________________________
systemd-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Mathieu Bridon
2012-02-29 04:09:37 UTC
Permalink
Post by David Lambert
I have got a little further in diagnosing this "memory leak" of
systemd-journal. It appears to worsen when I have a process that emits
data to stdout. I presume that systemd-journal is attempting to put this
somewhere but failing, and consequently is buffering it in its own
process.
Do you have the folder /var/log/journal on your system?

If it exists, systemd-journald will write logs there, otherwise it will
write them to /run/journal

And since /run is mounted as tmpfs, that could explain the memory usage
you are seeing?
--
Mathieu
David Lambert
2012-02-29 12:00:26 UTC
Permalink
Post by Mathieu Bridon
Do you have the folder /var/log/journal on your system?
If it exists, systemd-journald will write logs there, otherwise it will
write them to /run/journal
And since /run is mounted as tmpfs, that could explain the memory usage
you are seeing?
Thanks Mathieu - that was the issue! Two more questions, however:

1) What is the preferred method for turning off logging, or at least
redirecting it to /dev/null? I have an embedded system with very limited
resources.

2) What is the best resource for systemd documentation? For example, I
still have not found a man page for systemd-journald.conf


Dave.
Lennart Poettering
2012-02-29 13:26:07 UTC
Permalink
Post by David Lambert
Post by Mathieu Bridon
Do you have the folder /var/log/journal on your system?
If it exists, systemd-journald will write logs there, otherwise it will
write them to /run/journal
And since /run is mounted as tmpfs, that could explain the memory usage
you are seeing?
1) What is the preferred method for turning off logging, or at least
redirecting it to /dev/null? I have an embedded system with very
limited resources.
Turning logging off entirely is currently not supported, but it probably
makes sense to make this possible. I have now added this to the TODO
list.

You can put limits on the log size however, for example via
SystemMaxUse=64M or SystemKeepFree=500M or suchlike.

Note that there is currently a minimum file size of 64K enforced. (And
if something stores an entry larger than this the disk space consumption
might be beyond 64K temporarily)
Post by David Lambert
2) What is the best resource for systemd documentation? For example,
I still have not found a man page for systemd-journald.conf
Well, there's a huge body of documentation, for example, all the man
pages here:

http://www.freedesktop.org/software/systemd/man/

Or the blog stories which are linked from the systemd home page (look
for systemd for Administrators):

http://www.freedesktop.org/wiki/Software/systemd

However, I haven't found time to document the journal yet. Will do
before F17 is released.

Lennart
--
Lennart Poettering - Red Hat, Inc.
David Lambert
2012-03-03 17:29:17 UTC
Permalink
Post by Lennart Poettering
You can put limits on the log size however, for example via
SystemMaxUse=64M or SystemKeepFree=500M or suchlike.
I am still getting large logs even when I set SystemMaxUse=1M, as shown
below. Do I also need to set SystemMaxFileSize?

***@argus2:~# du -sh
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
14M /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal


***@argus2:~# cat /etc/systemd/systemd-journald.conf

# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# See system-journald.conf(5) for details

[Journal]
#Compress=yes
#RateLimitInterval=10s
#RateLimitBurst=200
SystemMaxUse=1M
#SystemKeepFree=
#SystemMaxFileSize=
#SystemMinFileSize=
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
#RuntimeMinFileSize=
ForwardToSyslog=no
#ForwardToKMsg=no
#ForwardToConsole=no
#ImportKernel=yes
David Lambert
2012-03-03 18:03:15 UTC
Permalink
Post by Lennart Poettering
You can put limits on the log size however, for example via
SystemMaxUse=64M or SystemKeepFree=500M or suchlike.
Further experimentation seems to indicate that the first time a limit is
reached, the system trims the log file back, BUT subsequently it appears
to ignore all limits:

***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
252K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
256K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
296K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
300K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
68K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
<<<<<<<<<<<<<<<<<<journal file trimmed as expected
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
76K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
76K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
88K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
100K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
132K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
140K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
160K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
272K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
288K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
336K /run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
612K
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal<<<<<<<<<<<<<<<<journal
file continues to grow ......
...
***@argus2:~# du -s
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal -h
1.3M
/run/log/journal/bd4dce8541064e29b00e90f7234b3287/system.journal<<<<<<<<<<<<<<<<....and
grow until all memory is exhausted :(

Configuration file is:

***@argus2:~# cat /etc/systemd/systemd-journald.conf
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# See system-journald.conf(5) for details

[Journal]
#Compress=yes
#RateLimitInterval=10s
#RateLimitBurst=200
SystemMaxUse=100K
#SystemKeepFree=
SystemMaxFileSize=200K
#SystemMinFileSize=
RuntimeMaxUse=1M
#RuntimeKeepFree=
RuntimeMaxFileSize=300K
#RuntimeMinFileSize=
ForwardToSyslog=no
#ForwardToKMsg=no
#ForwardToConsole=no
#ImportKernel=yes

Lennart Poettering
2012-02-29 13:15:52 UTC
Permalink
Post by David Lambert
Please excuse if this is trivial, but I am a systemd newbie.
I am running systemd on Angstrom/Beaglebone. After a couple of days
I notice that the process systemd-journald had grown and was
ultimately killed by the kernel's OOM handler. I investigated the
# See system-journald.conf(5) for details
Man pages for the journal and its tools are still missing. I will make
sure to add them in time for F17.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Continue reading on narkive:
Loading...