Return to README CVS log

Jump to this file's LXR Page

Up to [CENS] / misc / emlog

Took out the call to schedule() that we were doing after a write to the
log, since it didn't seem to have any effect on the reliability of
streaming data but still ends up costing the calling process some time.

Banner including version number now emitted to kernel log when module
is inserted.  Added version number constant to header file.

General cleanup: more consistent use of macros, etc.

Now compiles cleanly with -Wall (why didn't I do this before?)

emlog -- the EMbedded-system LOG-device

Version 0.40, 13 August 2001

Author:   Jeremy Elson <jelson@circlemud.org>
Web page: http://www.circlemud.org/~jelson/software/emlog

--------------------------------------------------------------------------


What is emlog?
==============

emlog is a Linux kernel module that makes it easy to access the most
recent (and *only* the most recent) output from a process.  It works
just like "tail -f" on a log file, except that the storage required
never grows.  This can be useful in embedded systems where there isn't
enough memory or disk space for keeping complete log files, but the
most recent debugging messages are sometimes needed (e.g., after an
error is observed).

The emlog kernel module implements simple character device driver.
The driver acts like a named pipe that has a finite, circular buffer.
The size of the buffer is easily configurable.  As more data is
written into the buffer, the oldest data is discarded.  A process that
reads from an emlog device will first read the existing buffer, then
see new text as it's written, similar to monitoring a log file using
"tail -f".

Version 0.30 of emlog (released March 1, 2001) should work under just
about any Linux kernel in the 2.x series, including 2.4.

emlog is free software, distributed under the GNU General Public
License (GPL); see the file COPYING for details.


How is emlog used?
==================

1: Configure, compile, and install emlog

   First, decide which major number you would like to use for emlog.
   This is configured in emlog.h using the constant
   EMLOG_MAJOR_NUMBER.  The default is 241, which is in the
   "local/experimental use" range according to the kernel
   documentation (similar to the 10/8 or 192.168/16 IP networks).
   Setting the major number to 0 will cause the kernel to dynamically
   assign a major number to emlog.

   Next, compile using the Makefile provided.  Typing 'make' should
   generate a single object file, 'emlog.o'.  Insert the module into
   the kernel using the 'insmod' command; e.g. 'insmod emlog.o'.

2: Create device files for emlog

   Next, you must use 'mknod' to create device files that your
   processes can write to.  The major number of the device files
   should be whatever number you selected in Step 1 (e.g., 241).  The
   minor number is used to indicate the *size* of the ring buffer for
   that device file, specified as the the number of kilobytes (e.g.,
   1024 bytes).  For example, to create an 8K buffer called 'testlog':

   % mknod /tmp/testlog c 241 8

   You can create as many devices as you like.  Internally, emlog uses
   the file's inode number to identify which buffer it refers to.

3: Write to and read from your new device file

   Once the device file has been created, simply write to your device
   file as you would any normal named pipe, e.g.

   % echo hello > /tmp/testlog

   Writes will never block because the buffer never runs out of space;
   old data is simply overwritten by new data.

   You can read from the log in the normal way, e.g. using cat.  Note
   that reads block, just like "tail -f", waiting for new log data.
   For example:

   % cat /tmp/testlog
   hello  [we immediately see the hello that we wrote in the previous step]
   _      [... and here's the cursor.  the 'cat' process is now
           blocked, waiting for new input.  New data will be displayed
           as it is written to the device by other processes.]
   ^C     [use control-c, for example, to stop reading.]

   Note that the first process to consume data from an emlog buffer
   will remove that text from the buffer.  This differs from, say,
   the behavior of the 'dmesg' buffer, which may be read many times
   until it scrolls off.

4: Remove emlog when you're done

   Type 'rmmod emlog' will remove the emlog kernel module and free all
   associated buffers.  This won't work until all emlog device files
   are closed.



Other Usage Notes
=================

emlog will allocate a fixed-size buffer on behalf of a device file if
one of the following two conditions is true:

  1-  A process has the file open for reading or writing
  2-  A process has written text to the pipe that has not been read

In other words, buffers are persistent, even after a process closes
the pipe.  If another process later reads the pipe, the text will
still be there.  Note that it is possible (naturally) to fill virtual
memory by creating many such pipes, writing to all of them, and never
reading the data out of them.  All buffers will be freed when the
emlog kernel module is removed.

Non-blocking reads work; i.e., setting O_NONBLOCK using ioctl() will
cause an EAGAIN to be returned if there is no data ready.  In
addition, the select() and poll() functions will work correctly on
emlog devices.


Emlog and devfs
===============

I love devfs and use it extensively, but I don't think it makes much
sense to use emlog with devfs.  emlog lets you create as many log
devices as you like, anywhere on the filesystem -- the module tells
them apart based on their inode number.  Having a single log device
always exist in a single place (/dev) is much less useful.  So, I have
intentionally continued using the old register_chrdev interface
instead of using the new devfs_register_chrdev interface.

 
Troubleshooting
===============

Q:  When I try to compile emlog, I get hundreds of errors related
to header files.

A:  If you've recently installed new kernel sources, make sure that
you've run "make config" or "make menuconfig" in /usr/src/linux.  You
don't actually have to go through the entire configuration; just make
sure that you have a /usr/include/asm and a /usr/include/linux that
are symbolic links into your kernel source tree.


Q:  When I try to insert the module using 'insmod', I get 'I/O error'.

A:  That probably means the major device number being registered by
emlog is already in use by another device driver.  Try changing the
major device number in emlog.h (or, change it to 0 in order to get a
dynamically assigned major number).


Q: I'm seeing "I/O error" at a time *other* then when the module is
inserted.

A:  Oops - you've found a bug in emlog.  Please report it.


Q:  When I try to access an emlog device file for reading or writing,
I get the error "no such device".

A:  This probably means either that the emlog kernel module is not
loaded; or, that the major number of the device file does not match
the major number that emlog registered.  To see which major number is
being used by emlog, type 'cat /proc/devices | grep emlog'.


Q:  When I try to access an emlog device file for reading or writing,
I get the error "invalid argument".

A:  The *minor* number of the emlog device file must be a number
between 1 and 128, representing the number of kilobytes (1,024 bytes)
that should be used for emlog's ring buffer.  Make sure you're
specifying a valid minor number in your 'mknod' statement.  Don't use
0.


Q:  I see "no memory" errors when I try opening new emlog files.

A:  Looks like you're out of virtual memory, sport.


Q:  When I try to remove the emlog driver ("rmmod emlog"), I get the
error "Device or resource busy".

A:  That means a process is currently using an emlog device.  You have
to wait until all processes close all emlog device files until the
driver can be removed.  Try using "lsof" to see which files are in use
by which processes.


Q:  When more than one process tries to read data from an emlog device,
only one gets it.

A:  Yep, that's how it's supposed to work -- it differs from dmesg.
Unlike dmesg, emlog lets you block waiting for more input (like tail
-f).  This was easier to implement by having buffers be consumed the
first time they were read.


Q:  You've made my computer crash.

A:  Sorry.  If you can reproduce the problem I'll try to fix it.


Known Bugs
==========

emlog identifies buffers based solely on the inode number of the
device file being accessed.  If two device files on two different
filesystems happen to have the same inode number, they will share the
same buffer, as if they were the same device file.

Bug reports, patches, complaints, praise, and submissions of Central
Services Form 27B/6, are welcomed by the author (Jeremy Elson,
<jelson@circlemud.org>.


Version History
===============

Version 0.30 (March 1, 2001)
 - Now compiles correctly for 2.4 series kernels.
 - select() and poll() now work correctly on emlog devices.
 - Bug fix: all instances should not share one wait queue!

Version 0.20 (June 14, 2000)
 - Initial public release.


Who wrote emlog, and why?
=========================

Emlog was written by Jeremy Elson <jelson@circlemud.org> at the
University of Southern California's Information Sciences Institute as
part of the SCADDS project <http://www.isi.edu/scadds>.  SCADDS is an
embedded systems research project.  We use small PC/104-bus-based
single-board-PCs using Linux.  We wanted to save the debugging output
from certain processes, but since these things have 16MB of disk space
and 32MB of RAM, keeping complete log files was not an option.  These
tiny nodes do have serial ports running PPP, though, so it's possible
to walk over to a node with a laptop, plug in a serial cable, and then
telnet into the box.  Using emlog, we can always keep the most recent
debug messages from our processes; in case of an error, we can plug in
a debug console and see what went wrong.

This work was supported by DARPA under grant No. DABT63-99-1-0011 as
part of the SCADDS project, and was also made possible in part due to
support from Cisco Systems.