Skip to content

spatial.join silently returns 0 rows if a point has 'NaN' coordinates (arcpy engine) #2556

Description

@joesweet-pima

Describe the bug
One point with 'NaN' coordinates makes spatial.join return 0 rows with no error when arcpy is the geometry engine. full_extent also returns nan. With shapely as the engine, the same data raises an error. ESRI REST Service queries can return 'NaN' in geometries.

To Reproduce
Steps to reproduce the behavior:

import pandas as pd
import numpy as np
from arcgis.features import GeoAccessor
from arcgis.geometry import Point, Polygon

sr = {'wkid': 2868}

# 10 valid points and 1 empty point ('NaN' is how REST services return empty points)
points = [Point({'x': 100 + i, 'y': 100 + i, 'spatialReference': sr}) for i in range(10)]
points.append(Point({'x': 'NaN', 'y': 'NaN', 'spatialReference': sr}))
pts = pd.DataFrame({'ID': range(11), 'SHAPE': points})
pts.spatial.set_geometry('SHAPE')

# One polygon that contains all 10 valid points
box = Polygon({'rings': [[[0, 0], [0, 500], [500, 500], [500, 0], [0, 0]]], 'spatialReference': sr})
polys = pd.DataFrame({'PID': [1], 'SHAPE': [box]})
polys.spatial.set_geometry('SHAPE')

# Print spatial join with NaN points
print('Silent error: 0 joins')
print(pts.spatial.full_extent)
print(len(polys.spatial.join(pts, how='inner', op='intersects')))   # expected 10

# Remove NaN points
is_valid = pts['SHAPE'].apply(lambda g: np.isfinite(np.array([g['x'], g['y']], dtype=float)).all())
pts = pts[is_valid].reset_index(drop=True)

# Print spatial join with NaN points removed
print('Joins successful')
print(pts.spatial.full_extent)
print(len(polys.spatial.join(pts, how='inner', op='intersects')))   # expected 10

error:

Silent error: 0 joins
(nan, nan, nan, nan)
0
Joins successful
(100.0, 100.0, 109.0, 109.0)
10

Expected behavior
arcgis engine should ignore the 'NaN' geometries and complete the join or should provide an error like shapely does letting the user know the cause of the failed join.

Platform (please complete the following information):

  • OS: Windows
  • Chrome
  • arcgis 2.4.2, arcpy 3.6.1, shapely 2.1.2

Additional context
shapely engine error on the same spatial.join: shapely.errors.GEOSException: UnsupportedOperationException:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions