0
votes

I have a date as string and now I want to persist the same datetime in a timestamp column in Cassandra db. On cqlsh query the timestamp is shown as different from the actual one. I am parsing it using simple date formatter SimpleDateFormat("E MMM dd HH:mm:ss yyyy") and storing.

ex: String date : Wed Jan 15 14:29:00 2020, In db: 2020-01-16 05:50:40.000000+0000

Also sometime, I am getting this error: DateOverFlowWarning: Some timestamps are larger than Python datetime can represent. Timestamps are displayed in milliseconds from epoch.

1

1 Answers

1
votes

Caveat: I do not use Cassandra. But perhaps this will help.

Data types

As you seem to have learned, do not store date-time values as text when your database supports an appropriate date-time data type.

Text formats

And, FYI, your format for representing a date-time value is terrible. When serializing date-time values as text, use the standard ISO 8601 formats.

Moment versus date-with-time-of-day

Another problem: You are trying to track moments with your use of type timestamp in Cassandra, but your stored strings such as Wed Jan 15 14:29:00 2020 is not a moment. Without the context of a time zone or offset-from-UTC, we cannot know if that time of almost 2:30 PM is 2:30 PM in Tokyo Japan or 2:30 PM in Toledo Ohio US, two very different moments several hours apart.

If you know the intended time zone for that date and time-of-day, apply a ZoneId to get a ZonedDateTime. I will assume that UTC (an offset of zero hours-minutes-seconds) was intended.

OffsetDateTime odt = ldt.atOffset( ZoneOffset.UTC ) ;

See this code run live at IdeOne.com.

Database

If you have a driver for Cassandra that supports JDBC 4.2 and later, you should be able to submit this java.time object directly.

myPreparedStatement.setObject( … , odt ) ;

If not, you can get a count of milliseconds since the epoch reference of first moment of 1970 UTC.

long milliseconds = odt.toInstant().toEpochMilli() ;

milliseconds: 1579098540000

Going the other direction.

Instant instant = Instant.ofEpochMilli( milliseconds ) ;

getting this error: DateOverFlowWarning: Some timestamps are larger than Python datetime can represent

I can only guess. Somewhere in your toolchain is a Python library. I suppose that library is limited to using a 32-bit integer. But you need a 64-bit integer to represent the count of milliseconds since 1970-01-01T00:00Z for a Cassandra timestamp value.


Table of date-time types in Java (both legacy and modern) and in standard SQL


About java.time

The java.time framework is built into Java 8 and later. These classes supplant the troublesome old legacy date-time classes such as java.util.Date, Calendar, & SimpleDateFormat.

To learn more, see the Oracle Tutorial. And search Stack Overflow for many examples and explanations. Specification is JSR 310.

The Joda-Time project, now in maintenance mode, advises migration to the java.time classes.

You may exchange java.time objects directly with your database. Use a JDBC driver compliant with JDBC 4.2 or later. No need for strings, no need for java.sql.* classes.

Where to obtain the java.time classes?