Spring Batch job metadata persistence issue

Question

We use Spring Batch to write our jobs and it works absolutely fine. But recently we saw an issue with the truncation that lead to some questions as to how spring batch stores meta data about the job.

Each time a job runs spring batch stores the meta data about the job in the db. There are 2 tables: batch_step_execution_context and batch_job_execution_context that are saving the meta data in 2 columns short_context and serialized_context. Both these columns save the exact data but short_context has a limit of 2500 characters and so the data is truncated.

Why do we 2 columns for storing the exact data, if we get rid of short_context what implications can it on the jobs

Right now we use an in memory HSQL db for our jobs. One of our job threw the following exception:

org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [UPDATE BATCH_JOB_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE JOB_EXECUTION_ID = ?]; data exception: string data, right truncation; nested exception is java.sql.SQLDataException: data exception: string data, right truncation

More stack trace:

Caused by: org.hsqldb.HsqlException: data exception: string data, right truncation
    at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.types.CharacterType.castOrConvertToType(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.types.CharacterType.convertToType(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.StatementDML.getUpdatedData(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.StatementDML.executeUpdateStatement(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.StatementDML.getResult(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.StatementDMQL.execute(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.Session.executeCompiledStatement(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]
    at org.hsqldb.Session.execute(Unknown Source) ~[hsqldb-2.3.3.jar!/:2.3.3]

Is this specific to HSQL db, because it works fine for other jobs, i.e. it is truncating it properly

thanks

serialized_context is not 2500 variable length character field, short_context is serialized_context is a text field — sachin jain

Dean Clark Dean Clark · Accepted Answer · 2016-07-07T15:43:04

The serialized_context column should be a CLOB (or LONGVARCHAR in HSQLDB), not a 2500 variable-length character field. Check the DDL here. If you change the datatype in your HSQLDB database, you should not see any truncation.

Beyond that, I would caution you from serializing too much in the execution context as it can severely impact performance. What is it you're storing that's so big?

Spring Batch job metadata persistence issue

1 Answers