3
votes

Current logging threshold that is all about log level(TRACE, DEBUG, INFO, WARN, ERROR and FATAL) is not enough for me. I have a gigabytes of log written from third party libraries into ERROR category in emergency case. I do not want to turn of this logging cause I want to see this problem logs. Most of logs are continuously repeating stack traces. So I want a kind of appender which will

  • 1)Skip logs if threshhold(kb/sec) is reaсhed (I mean when we are writing to much logs - we might skip some) or
  • 2)Skip stacktraces printing if more then one(n) stacktrace was printed in period of time

Please suggest

2

2 Answers

1
votes

I've written something similar in the past (sends up to X mails per Y time). It'll give you a direction. The setters are not mandatory but they allow you to change the defaults via log4j.properties.

public class LimitedSMTPAppender extends SMTPAppender {

    private int limit = 10;           // max at 10 mails ...
    private int cycleSeconds = 3600;  // ... per hour

    public void setLimit(int limit) {
        this.limit = limit;
    }

    public void setCycleSeconds(int cycleSeconds) {
        this.cycleSeconds = cycleSeconds;
    }

    private int lastVisited;
    private long lastCycle;

    protected boolean checkEntryConditions() {
        final long now = System.currentTimeMillis();
        final long thisCycle =  now - (now % (1000L*cycleSeconds));
        if (lastCycle!=thisCycle) {
            lastCycle = thisCycle;
            lastVisited = 0;
        }
        lastVisited++;
        return super.checkEntryConditions() && lastVisited<=limit;
    }

}
2
votes

Log4j do not have such an appender out of box. But you may do following:

Add appenders for third party libraries which uses gigabytes of data ("I have a gigabytes of log written from third party libraries into ERROR category"). And configure your appenders such that they do not use such a large amount of storage.

  1. RollingFileAppender Configure this third party library appender as rolling file. After some time they will use oldest file and you can keep only latest logs.

  2. JDBCAppender This appender accepts usage of stored procedures. Write a stored procedure so that it will do what you want.

For your first requirement add a calculated column or calculate this value in stored procedure and add to to your table columns. Then control this value to decide if you need to insert this log line to database.

For your second requirement You can take hash value (MD5,SHA etc) of exception message. If this hash value exists in database table you may ignore to insert it. Or you may count how many of it exists and decide how to insert to database accordingly.

You do not need to use enterprise database for these purposes, you may use apache derby for example, then all of your logs stay in application server. I think this one,JDBCAppender, meets your requirements more closely.