0
votes

I have successfully imported geodata into a class that I have called AdministrativeArea, which contains - among others - a WKT field. This field contains the polygon coordinate points for longitue and latitude. I have declared it as a string. I would like to query the documents to find out which of my polygons/geographic areas are contained within a given bounding rectangle. To do that, I have created an index using the spatial feature. Something seems to be "off" with this index or the way I'm building it: either the index count is stuck on 0 and nothing further happens, or it starts to build but gets stuck on a random number, and I get an error message. Can someone help out ?

Thanks !

The code/error message:

"WKT": "POLYGON ((21.7515487670898 62.0147895812988,21.7420005798341 62.0149002075195, etc. etc.

public class AdministrativeArea_ByNameAndWKT : AbstractIndexCreationTask<AdministrativeArea>
    {
        public AdministrativeArea_ByNameAndWKT()
        {
            Map = AdministrativeAreas => from a in AdministrativeAreas
                                         select new
                                         {
                                             Name = a.NAME_1,
                                             WKT = a.WKT
                                         };

            Spatial(x => x.WKT, options => options.Geography.Default());
        }
    }

Failed to load routed module (viewmodels/database/status/indexing/indexStats). Details: Load timeout for modules: d3/d3

Here's how I look for areas:

 IList<AdministrativeArea> results = session
                  .Query<AdministrativeArea, AdministrativeArea_ByNameAndWKT>()
                  .Spatial(x => x.WKT, criteria => criteria.WithinRadius(6.0, 0.24380, 6.636))
                  .ToList();

...and the class definition:

 public class AdministrativeArea
    {
        public string WKT { get; set; }
        public int OBJECTID { get; set; }
        public float UID { get; set; }
        public int ID_0 { get; set; }
        public string ISO { get; set; }
        public string NAME_0 { get; set; }
        public int ID_1 { get; set; }
        public string NAME_1 { get; set; }
        public string VARNAME_1 { get; set; }
        public string NL_NAME_1 { get; set; }
        public string HASC_1 { get; set; }
        public string CCN_1 { get; set; }
        public string CCA_1 { get; set; }
        public string TYPE_1 { get; set; }
        public string ENGTYPE_1 { get; set; }
        public string VALIDFR_1 { get; set; }
        public string VALIDTO_1 { get; set; }
        public string REMARKS_1 { get; set; }
        public int? ID_2 { get; set; }
        public string NAME_2 { get; set; }
        public string VARNAME_2 { get; set; }
        public string NL_NAME_2 { get; set; }
        public string HASC_2 { get; set; }
        public int CCN_2 { get; set; }
        public string CCA_2 { get; set; }
        public string TYPE_2 { get; set; }
        public string ENGTYPE_2 { get; set; }
        public string VALIDFR_2 { get; set; }
        public string VALIDTO_2 { get; set; }
        public string REMARKS_2 { get; set; }
        public int ID_3 { get; set; }
        public string NAME_3 { get; set; }
        public string VARNAME_3 { get; set; }
        public string NL_NAME_3 { get; set; }
        public string HASC_3 { get; set; }
        public int CCN_3 { get; set; }
        public string CCA_3 { get; set; }
        public string TYPE_3 { get; set; }
        public string ENGTYPE_3 { get; set; }
        public string VALIDFR_3 { get; set; }
        public string VALIDTO_3 { get; set; }
        public string REMARKS_3 { get; set; }
        public int ID_4 { get; set; }
        public string NAME_4 { get; set; }
        public string VARNAME_4 { get; set; }
        public int CCN_4 { get; set; }
        public string CCA_4 { get; set; }
        public string TYPE_4 { get; set; }
        public string ENGTYPE_4 { get; set; }
        public string VALIDFR_4 { get; set; }
        public string VALIDTO_4 { get; set; }
        public string REMARKS_4 { get; set; }
        public int ID_5 { get; set; }
        public string NAME_5 { get; set; }
        public int CCN_5 { get; set; }
        public string CCA_5 { get; set; }
        public string TYPE_5 { get; set; }
        public string ENGTYPE_5 { get; set; }
        public string REGION { get; set; }
        public string VARREGION { get; set; }
        public float Shape_Leng { get; set; }
        public float Shape_Area { get; set; }
    }

...and an example JSON (there are ~260.000 of these...)

{
    "WKT": "POLYGON ((29.666404724121 -28.5651550292969,29.6635494232178 -28.5674591064453,29.6602096557618 -28.5701694488525,29.6568603515626 -28.5728702545166,29.6519393920898 -28.5768508911132,29.6490306854248 -28.5791893005371,29.6461200714112 -28.5815391540527,29.6373901367189 -28.58856010437,29.6344795227051 -28.5909099578857,29.6345596313476 -28.5909996032715,29.6346702575684 -28.5911197662353,29.6353797912599 -28.5918998718262,29.6361103057862 -28.5927104949951,29.6368408203124 -28.5935096740723,29.6375408172609 -28.5942897796631,29.6375904083253 -28.5943508148193,29.6382408142091 -28.5950794219971,29.6389408111573 -28.595890045166,29.6396198272705 -28.5966701507568,29.6403102874756 -28.5974502563476,29.6410102844238))",
    "OBJECTID": 207648,
    "UID": 20764800000000000,
    "ID_0": 211,
    "ISO": "ZAF",
    "NAME_0": "South Africa",
    "ID_1": 4,
    "NAME_1": "KwaZulu-Natal",
    "VARNAME_1": "Natal and Zululand",
    "NL_NAME_1": "",
    "HASC_1": "ZA.NL",
    "CCN_1": 0,
    "CCA_1": "KZN",
    "TYPE_1": "Provinsie",
    "ENGTYPE_1": "Province",
    "VALIDFR_1": "Unknown",
    "VALIDTO_1": "Present",
    "REMARKS_1": "",
    "ID_2": 27,
    "NAME_2": "Uthukela",
    "VARNAME_2": "",
    "NL_NAME_2": "",
    "HASC_2": "ZA.NL.UL",
    "CCN_2": 0,
    "CCA_2": "DC23",
    "TYPE_2": "District Municipality",
    "ENGTYPE_2": "District Municipality",
    "VALIDFR_2": 2011,
    "VALIDTO_2": "Current",
    "REMARKS_2": "",
    "ID_3": 108,
    "NAME_3": "Okhahlamba",
    "VARNAME_3": "",
    "NL_NAME_3": "",
    "HASC_3": "",
    "CCN_3": 0,
    "CCA_3": "KZN235",
    "TYPE_3": "Local Municipality",
    "ENGTYPE_3": "Local Municipality",
    "VALIDFR_3": 2011,
    "VALIDTO_3": "Current",
    "REMARKS_3": "",
    "ID_4": 2161,
    "NAME_4": 13,
    "VARNAME_4": "",
    "CCN_4": 52305013,
    "CCA_4": "",
    "TYPE_4": "Ward",
    "ENGTYPE_4": "Ward",
    "VALIDFR_4": 2011,
    "VALIDTO_4": "Current",
    "REMARKS_4": "",
    "ID_5": 0,
    "NAME_5": "",
    "CCN_5": 0,
    "CCA_5": "",
    "TYPE_5": "",
    "ENGTYPE_5": "",
    "REGION": "",
    "VARREGION": "",
    "Shape_Leng": 194782209549,
    "Shape_Area": 6270373326
}

By the way: there are hundreds, if not thousands of lat/lng points within each of the 260.000 areas (I cut off about 90% of the coordinates in the example above for space reasons...). Maybe my approach just isn't right for this case ?

The RavenDB indexing behaviour is very odd for this: it seems to get "stuck" on the indexing task (CPU is between 20% and 40$, and RAM consumption goes up consistently but slowly), but no new index entries are created, and no error messages appear. I am running this on a laptop with 16GB RAM and an i7 with ~3.6GHz.

1
Please post your actual document with the full data. - Ayende Rahien
@AyendeRahien I have updated the description. - Hiro Protagonist
Ok, perhaps I have seriously underestimated computing resource requirements for this geodata indexing task: it DOES progress, but mindbogglingly slow. I suppose that's understandable, as it has to build a hash (is that what it actually does ?) for millions of lat-lng coordinate points. @AyendeRahien do you have any - even remotely - comparable performance stats, so I can get an idea of what it would take to get this sorted out ? If the answer is: it'll take 2 months on your laptop, I'll have to consider other options :-) - Hiro Protagonist

1 Answers

1
votes

This is what your shape looks like WKT shape

The problem is that this shape is about 5 KM in length, and 600 - 700 meters in width, and it is not nicely aligned. The default precision mode that we have is set to about 5 meters, so that means that we need to effectively draw a shape that is composed of many small 5 meter pieces, a rough estimate would be about 350K of them.

Either simplify the shape or reduce the precision.

You can see the docs about spatial precision is here: http://ravendb.net/docs/article-page/3.5/Csharp/indexes/indexing-spatial-data