Logging Brainstorm

During the Sept. 2009 commitfest, at least two patches have come in to modify logging. After some discussion, both were rejected with the suggestion that this whole idea needs more discussion before it needs more coding.

Wishlist

This list includes things we might like to be able to do and currently can't.

Log actions in different databases to different files

syslog is the best destination for error logging, but the performance is bad when we writes a lot of logs. It would be better to split query and performance logs into text logs. Syslog-based filtering is not a solution because performance bottleneck is in the communication of syslog.

Log differently based on source database

i.e. make log_line_prefix, log_min_messages, etc. database-specific

Do either of the above with some criterion other than just database

e.g. different users' actions end up logged in different places

Provide logs that are both machine-readable and human-readable.

We've got csvlog, which sucks for humans to read, and other logs, which suck for machines to read. One possible solution is to allow logging to two files -- one in CSV and another in "plain text".

Add option to fix the latest log filename

Machine-monitoring will be easier if we could fix the latest log filename. The latest message is always at the tail of the same file. If the file size shrinks, monitoring middleware reopens the same filename. The log will be renamed on log rotation; timestamp for formatting is retrieved from creation time of the file.

Use unique server log encoding for databases with different encoding

We merge logs in different encoding into a file for now. The encoding of server log should be specified independent from database encoding. UTF-8 might be only reasonable encoding though.

Different localization in server logs and client messages

We use one lc_messages for both server logs and client messages, but it would be better to have different encoding in them. DBA and user could want different localization.

Additional fields to report object names in error message

The name and schema name of the relevant object can be pulled out of the message by programs without having to parse the message text.

NOT NULL violation and error-message

Implementation Idea

Use executor hooks for query logging

Query logging could be implemented in external modules and we could avoid including it in the core feature. We have ExecutorStart_hook in 8.4, but need additional ProcessUtility_hook to filter also DDL commands.

Logging hook (rejected)

Adding send_message_to_server_log_hook in elog and do all operations listed on the wishlist in user-defined modules. But having such critical behavior dependent on a loadable module is a bad idea [1].