Scala iterator.count for large lists (>int32 size)

Question

I have large lists that I need to get the size of. Count returns an int, according to the scala api https://www.scala-lang.org/api/2.13.3/scala/collection/immutable/List.html, which may overflow on large lists.

How do I get around this?

Is there actually a list of that size, or is this a hypothetical..? Counting a (linked) list is also O(n).. anyway, a long count value could be counted with fold/foldLeft. — user2864740
This is not hypothetical. I'm migrating a system to scala that will have up to 10 billion ids to keep track of and generate internal incrementing ids for I guess we have to use the manual way and not rely on the api — Samson Hu
That sounds like an awful lot to keep in memory. Also note that length is an O(n) operation. Perhaps a different structure / algorithm (not needing the length) might be more suitable. Good luck. — user2864740

gianluca aguzzi gianluca aguzzi · Accepted Answer · 2021-09-08T07:13:30

A solution could be:

aList.foldLeft(0L)((acc, _) => acc + 1)

This will return the total number of elements as a Long number. If you want, you can wrap this functionality with the Pimp my library pattern:

implicit class LongList[A](list: List[A]) {
  def sizeL: Long = list.foldLeft(0L)((acc, _) => acc + 1)
}

As underlying in the comments, this operation will cost you O(n), so if you have other alternatives try to explore them :)