How a presentation made a tool better...
The aim of doing a set of Dstat presentations was not just to promote Dstat, but also to hear people's impressions and ideas. And lots of ideas were shared and ended up in the TODO list.
But one of the questions that I did not quite understand initially, turned out to be an interesting and useful idea. And after the presentation we sat together to show how simple it was to bring an idea into practice.
The question was something like: can Dstat be used to understand how the OOM (Out-Of-Memory) killer works ? The question at first made no sense to me, since I did not understand what the OOM killer had in common with a performance monitoring tool.
But when I later was looking into another idea from another attendant, we noticed the /proc/pid/oom_score and /proc/pid/oom_adj files. And then it made all sense. The OOM algorithm adds a score to each PID and the higher the score the more prefered that process will be when the reaper calls.
So this was the perfect opportunity to write a new dstat_topoom plugin. It took about 30 minutes to write a plugin that displays the process that is the most likely to be killed by the OOM killer.
Not only is this useful to see how this OOM score relates to the memory usage, IO usage or CPU usage of the process. It can also be used pro-actively to see what process(es) are in danger of being killed by the OOM killer and this is useful knowledge if you want to prevent the system to make a bad decision. Setting oom_adj to -17 will prevent that PID to be shot in the head.
This goes to show how giving a presentation about a tool you wrote to help yourself in a specific case, can be used by someone who is in a complete different situation but recognizes the tool as being an aid to fix his problem as well.
What's more, I now have an extra use-case that I can show off in my next Dstat presentation :-)