SAP Hybris logs to Application Insights are very large(more than 100 GB per day for Production)

0
votes

We are using Application Insights for logging for all the microservices which work fine and data is not that too large. But when We send our SAP Hybris Commerce logs(6.7 version) it is sending the humongous amount of logs to Application Insight(more than 100 GB) for INFO level which is creating huge costs just for logging. Please provide some ways to reduce the not useful Hybris logs to be sent to Azure Application Insights. Please suggest what can be the best solution for this.

1
loggingazure-application-insightshybris
Consider monitoring less. Why not just log those events outside a given range, which means all events within spec are not recorded. - Solar Mike
These are events which are directly being appended from log4j2 to Application insight Appender and sent to Application Insight. We changed the log level to Error. However, we are still getting 40-50 gb of logs on a daily basis which costs 150 dollars a day. Can you please explain how to not log these log events based on what criteria? - Master

1 Answers

2
votes

I do not see any reason why you should keep the logging level of INFO in production. As you can see the in the image below, Log4j differentiates five such log levels (in ascending order): DEBUG, INFO, WARN, ERROR, FATAL, NONE.

enter image description here

You should change the logging level of the classes to ERROR which will enable only ERROR and FATAL to be captured. In production, at most it should be WARN which will enable only WARN, ERROR and FATAL logs to be captured.

How to change log level?

There are two ways to do it:

  1. Temporary: Login to hAC (Hybris Admin Console) ▸ ⁨Platform ▸ ⁨Logging and search the fully-qualified name of a class (e.g. de.hybris.platform.jalo.flexiblesearch.FlexibleSearch) and then change its logging level. This change will remain effective until you restart the server. Another way to do it temporarily is by using scripting but it requires knowledge of the groovy script.

  2. Permanent: Put the configurations into the local.properties file. Note that it will require restarting the server to become effective. A sample configuration:

    log4j.logger.my.package = WARN
log4j.logger.org.training.service.MyServiceImpl = ERROR

Further reading: https://help.sap.com/viewer/d0224eca81e249cb821f2cdf45a82ace/6.7.0.0/en-US/8c07853c866910148a00baf81ea1669e.html