ArangoDB 3.8 crashing on simple queries

Question

I hope someone can point me in the right direction. I have avidly used ArangoDB since 2017 and have developed a model that works well for exploring longitudinal analysis of student study. However, since moving to RocksDB and 3.8, I have started to experience crashes and cannot work out why. I'm using the Community edition on my Windows 10 laptop with 16GB RAM. I don't think the computer is the problem as I also experience it on my desktop machine. Also, Windows 10 with 32GB RAM. My database uses two collections: a document collection, 'Study' that contains details around a student's study attempt at a module with 1.5 million documents; an edge collection, 'Path' that contains the links between study attempts, with 2.8 million edges; < 6 edges per student on average. The StartEnd document collection has a pair of dummy start and end vertices for each student. These make it easy to produce the study path for each student. There are just under a million documents. Simple queries that used to run without issue without limits, now crash arangoDB if I do not limit the number of results to around 10. For example, this query selects a number of students with LIMIT, collects their study attempts (vertices), just keeping minimal data to RETURN. The objects retained are pi, an 8 character student id; pres5, a year plus letter, e.g. 2012J; module, a string of between 4 and 7 characters.This query would run without issue on previous versions in MMfiles:

FOR id in temp2  //list of user ids 
  LIMIT 5,10 //have tried various combinations of values within LIMIT 
  FOR v,e,p IN 1..10 OUTBOUND CONCAT("StartEnd/",id._key,"-start") Path  
    OPTIONS {"bfs":true, uniqueVertices :"global", ignoreErrors:true}    
    FILTER !IS_NULL(v.module) //removes the final vertex for each user, as always null    
    LET summary=KEEP(v,"pi","pres5", "module")   //retain 3 objects, id, presentation, module 
  RETURN summary

When I look at the AragnoDB log for this failure. The entry is this:

2021-11-07T21:43:44Z [18988] INFO [144fe] {general} using storage engine 'rocksdb'
2021-11-07T21:43:44Z [18988] INFO [3bb7d] {cluster} Starting up with role SINGLE
2021-11-07T21:43:44Z [18988] INFO [f6e0e] {aql} memory limit per AQL query automatically set to 10165316813 bytes. to modify this value, please adjust the startup option --query.memory-limit
2021-11-07T21:43:50Z [18988] INFO [c1b63] {arangosearch} ArangoSearch maintenance: [1..1] commit thread(s), [1..1] consolidation thread(s)
2021-11-07T21:43:50Z [18988] INFO [6ea38] {general} using endpoint 'http+tcp://127.0.0.1:8529' for non-encrypted requests
2021-11-07T21:43:51Z [18988] INFO [cf3f4] {general} ArangoDB (version 3.8.1 [win64]) is ready for business. Have fun!
2021-11-07T21:44:46Z [18988] INFO [87ff4] {crash} Unhandled exception: c0000005 at address 00007ff7f761e02e in thread 5ec
2021-11-07T21:44:47Z [18988] INFO [93315] {crash} Wrote minidump: C:\WINDOWS\TEMP\arangod_a15316\2021-11-07T21-44-46_18988_1516.dmp
2021-11-08T15:35:58Z [9048] INFO [e52b0] {general} ArangoDB 3.8.1 [win64] 64bit, using build tags/v3.8.1-0-g20aa2c22713-dirty, VPack 0.1.35, RocksDB 6.8.0, ICU 64.2, V8 7.9.317, OpenSSL 1.1.1k 25 Mar 2021

The original query I was working on was a little more complicated, with a COLLECT statement to collect some values around pi and pres5 (as some students study 2/3 modules in the same presentation. Again this query would run without issue in previous ArangoDB installation but now causes ArangoDB to crash like this:

2021-11-06T20:11:29Z [66548] INFO [144fe] {general} using storage engine 'rocksdb'
2021-11-06T20:11:29Z [66548] INFO [3bb7d] {cluster} Starting up with role SINGLE
2021-11-06T20:11:29Z [66548] INFO [f6e0e] {aql} memory limit per AQL query automatically set to 10165316813 bytes. to modify this value, please adjust the startup option --query.memory-limit
2021-11-06T20:11:36Z [66548] INFO [c1b63] {arangosearch} ArangoSearch maintenance: [1..1] commit thread(s), [1..1] consolidation thread(s)
2021-11-06T20:11:36Z [66548] INFO [6ea38] {general} using endpoint 'http+tcp://127.0.0.1:8529' for non-encrypted requests
2021-11-06T20:11:38Z [66548] INFO [cf3f4] {general} ArangoDB (version 3.8.1 [win64]) is ready for business. Have fun!
2021-11-07T07:44:23Z [66548] INFO [3278a] {general} system-shutdown received, beginning shut down sequence
2021-11-07T07:44:23Z [66548] INFO [63d54] {queries} AQLFeature shutdown, waiting for 0 registered queries to terminate and for 1 feature leases to be released
2021-11-07T07:45:22Z [5968] INFO [e52b0] {general} ArangoDB 3.8.1 [win64] 64bit, using build tags/v3.8.1-0-g20aa2c22713-dirty, VPack 0.1.35, RocksDB 6.8.0, ICU 64.2, V8 7.9.317, OpenSSL 1.1.1k 25 Mar 2021

Many thanks. Chris

Answer 1

If you read release notes for 3.8 there is a mention of the new memory limit options, you are able to override those.