I had a fun idea one day after seeing some people online playing with receipt printers. I frequently run into app crashes on my Linux system, all of which generate core dumps for review later, and those are logged in the system log. Wouldn't it be funny if they were logged not only in software, but also in physical form? It took me a while to finally go through with it, but I now have a receipt printer set up to automatically print out a crash log for every crash on my system. Here's how I did it.
systemd-coredump
On Linux systems using systemd (which is the vast majority of them), a component called systemd-coredump is installed to automatically take in coredump files that the kernel generates, and processes the files before outputting them on disk. Coredump files are stored in /var/lib/systemd/coredump
by default, and the coredumpctl
command can be used to output various information about the coredumps. (This will be useful later.)
My first task was to get the system to run a command whenever a coredump occurred. I spent some time digging through man pages on systemd-coredump and coredumpctl, but I unfortunately came up with nothing - it doesn't seem to have any functionality to e.g. run a service on crash. I tried hacking with the inner functionality of the dump service, trying to hook into its trigger, but this didn't result in anything useful. I also thought about replacing systemd-coredump entirely, but then I'd be lacking the digest that it creates.
Then I came up with a solution...
Watching directories with systemd
I realized that I could simply have a command be triggered whenever the coredump directory gets written to, which only happens when a program crashes. systemd supports special unit files with the extension .path
to watch a specific file or directory. When the path is updated, systemd triggers a .service
unit with the same name, which can be a shell script with the commands I need. Perfect!
I put together a simple path unit /etc/systemd/system/print-coredump.path
:
[Path]
PathChanged=/var/lib/systemd/coredump
[Install]
WantedBy=paths.target
And I made an accompanying service unit to run a bash script:
[Unit]
Description=Automatically print core dump
[Service]
Type=oneshot
ExecStart=/usr/local/bin/print-coredump
To enable watching, I simply had to run sudo systemctl enable --now print-coredump.path
.
ReceiptIO
Next I needed a way to send text to the printer. I went googling around for a bit, and eventually stumbled upon ReceiptIO. ReceiptIO is a NodeJS library and program that automatically formats and prints out text (and images) using its own formatting language inspired by Markdown. It also has a web viewer to preview printouts, which was useful before I had the printer in my hands, and it also helped me understand what changes I'd need to make it pretty.
I installed it globally with npm: sudo npm install -g receiptio
To start with, I made a very simple bash script to print out the most recent coredump generated to an SVG (the default output if no printer is attached):
#!/bin/bash
sleep 1 # wait for the file to be fully written
coredumpctl info -1 | receiptio -i -o /home/jack/printtest.svg
As I expected, the output it generated was nowhere near formatted in a way that was readable. So I then spent some time writing some commands to process the coredump digest into a nicer format.
Formatting the output
I started by plugging in the output of coredumpctl info
straight into the web viewer. My first challenge was that all of the text was centered horizontally - this looked really ugly. At first, I simply made all of the text align left, but the header info at the top didn't quite look right.
One thing that coredumpctl
does in its output is that it formats the metadata block at the top in two columns using spaces. However, this doesn't translate well to the receipt printer, so my output was overflowing the lines, and it just didn't look good. To fix that, I used the column formatting commands that ReceiptIO provides. I used sed
to replace every a: b
with a:|b
- |
tells ReceiptIO to separate the sides into columns, and having no spaces on either side of it means that the left side is right aligned, and the right side is left aligned (i.e. the text is effectively centered). One caveat: I did decide to undo this change on the final line, which I wanted to keep centered - this involved another sed
expression to change it back if the left side was architecture
.
After that, I needed to do some formatting on the stack trace entries. I wanted to keep the "introduction" to each thread centered, but each frame would be left-aligned. This simply involved adding a |
at the start of each frame line, which conveniently always starts with #
. A simple sed
expression fixed that. I also made it replace the empty lines with horizontal rules (-
alone on a line) for further prettification.
My last gripe was that the underscores in function names in the traces were being interpreted as control characters - like in Markdown, _a_
and __a__
apply special formatting. This required one last sed
expression to escape every _
in the output.
Building the command
Next I combined the changes into a single command line. Watch out, this is a pretty big command!
coredumpctl info -1 | sed -e 's/^ *//g' -e 's/: /:|/g' -e 's/^#/|#/g' -e 's/_/\\_/g' -e 's/architecture:|/architecture: /' -e 's/^$/-/g' | receiptio -d /dev/usb/lp0 -c 42
Let's break that really long sed
line down into each part:
- If you aren't aware of how
sed
works: basically, it's a simple text processing language that transforms each line of text passed into it via a pipe. The most basic command, and the only one used here, iss
for substitution, which takes a regular expression pattern and replaces it with other text. It takes the syntaxs/find/replace/flags
, whereflags
may be empty to replace only once, or it can beg
to replace all. - Each
-e
option is a single replacement command. This makes it so I don't have to runsed
a bunch of different times - it can be done all in one command. - The first
-e
command gets rid of any spaces at the start of a line, which basically removes the column formatting thatcoredumpctl
applies. - The second
-e
command replaces all:
s with:|
, placing metadata lines into two columns as described above. - The third
-e
command left-aligns all lines starting with#
(stack frame lines). - The fourth
-e
command escapes all_
s with\_
(double backslash sosed
doesn't see it as an escape itself). - The fifth
-e
command removes the column formatting from the final line (witharchitecture:
in it). - The last
-e
command finds all empty lines and replaces them with a single-
, which indicates a horizontal rule.
After that behemoth, the output is sent to receiptio
, which prints to the printer over USB, with 42 columns of characters (the spec of my printer).
And sure enough it worked!
Further tweaking
One of the main things that bothered me about this print was that there was so much whitespace on the left side that wasn't being used, which could've been used for more text on the right. After looking through the docs, I found that I could use a special command, {w:}
, to set the widths of each column to a specific value.
I played around a bit in the web viewer, and settled on the command {w:15 *}
, which makes the left side 15 characters wide, and makes the right side the rest of the width (27 in my case). But I had one problem - the stack traces were now limited to being 15 characters wide, so I needed to reset the columns with {w:*}
after the header. This looked like a tough problem at first, since sed
only sees the file as independent lines. However, I was able to fix it by making the command piggyback off of the horizontal rule replacement - in other words, now it would replace empty lines with both {w:*}
and -
.
Prepending the {w:15 *}
command appeared tough at first, but with some internet digging, I managed to come up with this command:
{ echo "{w:15 *}" ; coredumpctl info -1 | sed -e 's/^ *//g' -e 's/: /:|/g' -e 's/^#/|#/g' -e 's/_/\\_/g' -e 's/architecture:|/architecture: /' -e 's/^$/{w:*}\n-/g' ; } | receiptio -d /dev/usb/lp0 -c 42
This uses piping on two combined commands: first an echo
, and then the coredump command, now with the {w:*}
commands filled in. This worked fine, but I soon had to swap it out for a better solution.
Next I added a little logo to the top - it was kinda boring with just text. I took the monochrome Linux logo from the kernel, scaled it up, and added a !
after it to indicate a crash. Then I saved it as a monochrome PNG. ReceiptIO can display an image using the {i:}
command, which takes base64 of a PNG file as an argument. I didn't want to shove all this into the same command line, so I split it off into a separate file /usr/local/share/coredump-header.txt
:
{i:iVBORw0KGgoAAAANSUhEUgAAAWgAAADwCAAAAAAIzKGTAAAACXBIWXMAAAsTAAALEwEAmpwYAAAEWElEQVR42u3d23KjMBCE4WjL7//Ksxdbu8XGhgh0QBJf38VxsOtPq5lBEqT4oh76BQHQQBPQQANNQANNp/Xq+WFp5/XgaAJ6MqU+wzZlvCc4moAWHSdzY/kA4WigRUe73Fg4PTgaaNFRPTfi7c3B0QT00HoN8j3S24/B0QS06Kjd1HA0Ab1qdCR0ORpo0XFHKRIcTes4eryiPji6phrlFdAyGmgCGmjlXbfzeDo8v6fVexaOBlp0XKv/43DlRqx+WZWjgRYdZ5X+hkN8Wr/xNUtpERwtOqhfdKSMX6WdYOFoAvrJVcc2HziagJ6qXr4/Or5VIE/bhs/RQKs6zibewQWNs2HC0QT0Q6Jj26ekzURt/J8nJmcJaNGx13TEfidicpaA7qvC0/JAO2fNsBDQqo7MpmN7GWTtnoWjgRYdVdJjpsVgHA30k1uTWxuWnA0s24ZlveaFo4FWdZwNvIOmY6/2SJv0WKNn4WigRcfZMuNjemhYCGjRkVl7/Nh+xf8bW74VIRxNQNcck+NFRyr4prHoVAtHA63q6BdsHE1APyY62s2MTF14cDTQ8zYnHaKjyhPcDmZm500PjgZa1dG/3lggPTgaaNHRp8CyJIyAfkJ0pJ1X3rfVR8afx3KFB0d36sqB7qQ6T7rPnI1NebmRWXg0TI/E0aoOuqdhiTOD/fiGPxxNQC9ZdVQ/RecHSKvCQ9UxS3sC9EJVR1Q6gsukBPTC0dFugL8HyMcnQAVHE9DzVx3VB/Iys7QcDbTo6FBy7H3cvIUHR3c69wA9YdVhsxtHAy06Ris5OJqAXrXqaFfDpvl7Fo7uVKkCzdFAE9DLVR1n10ylx18J4WiggSaggQYagj4dOdAcDTQBDfTjNcEMy/sTW2bc5MLRHD175Twj6Ni5MDtReogOoNfSlU33p27/VVhs3JiqHC06qGLV8bF9aPRw2JxbAM3SvHA00E+Ojr0HxVZPj8zjBEcT0MNHx/EDpvs/mD44moAeODryn8q0TY+6QztO3jKdo0UH3Rgdhdc/U+306Ky6l3/NGf58Zko1UAP9E+ZKqF91B2x890K/IV/xs9KnlwoP7mSYxfmr+JI30LWJ3pXRfdLjT2RVaZR2OZfdPJ6jO1kd6C7B0eiO6LdMlf6bpR1zly5Hd3I70J30ajSK71poEYXp0ex7c7SMFh2DV1hjLlDnaKBFR/74vbzKnaMJ6IqN/FxVx4X0SOv+FzhadKg6WqRHxdAoO1SkNvnN0aJjtDNeWUFy87WO1OZoRUwicfSdnraApgvpKO1kXh2+dKplqozcqNPZvaVH+WEtcvzZHpbtDttvLwq60Q1DVB0aFrocHeM/BKH6VjuOFh1UGB3b1d0DDs8pig2OBnpR5W7MuLznPZ38q8xAmKXY4GigHx4dl2Pk8sKwWCg3OBroRzUsB43MdhQfDOQW05wz5gZHA63qyKlDrrU2+ZkTHE1ALxEdxNFAA01AA01AAw00NdBvS6PYP/o0f6cAAAAASUVORK5CYII=}
{w:15 *}
Then I updated the command to concatenate that file and stdin
from the coredumpctl
/sed
chain:
#!/bin/bash
sleep 1
coredumpctl info -1 | sed -e 's/^ *//g' -e 's/: /:|/g' -e 's/^#/|#/g' -e 's/_/\\_/g' -e 's/architecture:|/architecture: /' -e 's/^$/{w:*}\n-/g' | cat /usr/local/share/coredump-header.txt - | receiptio -d /dev/usb/lp0 -c 42
This produced my final output:
Oops...
While writing this blog, I had a little accident. plasmashell, the KDE desktop manager, crashed, and successfully demonstrated that the script worked! But...
Yeah, uh, plasmashell has a LOT of threads. 220" (555cm) of threads, in fact. It was hilarious to watch it keep printing for a solid 20 seconds straight (this printer is fast!), but I don't need a useless piece of paper that's longer than the room I live in, especially when I get these sorts of crashes every day. To safeguard, I made my final edit: stopping the output after 100 lines (should be reasonable).
#!/bin/bash
sleep 1
coredumpctl info -1 | sed -e 's/^ *//g' -e 's/: /:|/g' -e 's/^#/|#/g' -e 's/_/\\_/g' -e 's/architecture:|/architecture: /' -e 's/^$/{w:*}\n-/g' | cat /usr/local/share/coredump-header.txt - | head -n 100 | receiptio -d /dev/usb/lp0 -c 42
Conclusion
This was probably a complete waste of my time and nearly $100 in acquiring the receipt printer and paper. But it was a really fun time to put it together, especially when I cried laughing at the sheer length of that plasmashell crash. It also taught me some things about receipt printers and coredumps. Maybe someone will decide it's worth it to set this up, which is why I'm putting this post out there. Anyway, remember folks: if you're gonna accidentally print over 6 yards of useless receipts, it had better not be coming out of your pocket.
Top comments (0)