we try to use Titan (1.0.0 version) with DynamoDB backend like our recommendation system engine. We have a huge user’s database with their relationships. It contains about 3.5 millions of users and about 2 billions of relationships between users. Here is the code that we used to create schema
https://gist.github.com/angryTit/3b1a4125fc72bc8b9e9bb395892caf92
As you can see we use one composite index to find starting point of traversal fast, 5 edge’s types and some properties.
In our case users can have a really big amount of edges. Each could have tens of thousands edges.
Here is the code that we use to provide recommendations online
https://gist.github.com/angryTit/e0d1e18c0074cc8549b053709f63efdf
The problem that the traversal works very slow. This one
https://gist.github.com/angryTit/e0d1e18c0074cc8549b053709f63efdf#file-reco-L28
tooks 20 - 30 seconds in case when user has about 5000 - 6000 edges.
Our DynamoDB’s tables has enough read/write capacity (we can see from CloudWatch that consumed capacity lower than provided by 1000 units.)
Here is our configuration of Titan
https://gist.github.com/angryTit/904609f0c90beca5f90e94accc7199e5
We tried to run it inside Lambda functions with max memory and on the big instance (r3.8xlarge) but with the same results...
Are we doing something wrong or it's normal in our case?
Thank you.
