Getting new data

[wfengk]

Need feedback

So right now we are at about ~1000 restaurants, and we will need to get more data eventually.

I was thinking that each time we use the search to find new restaurants, we could query yelp and add that new restaurant to our database

Thoughts?

[RedTn]

Hmm, it seems like the crawler should be doing this. From a technical standpoint, other than the 3 RTT calls for each missing restaurant (Client -> Ask Toni, Client -> Yelp, Client -> Ask Toni), the comparison logic is going to be way too complex. Restaurant with same names, then compare address, what if we want to overwrite AskToni version?

[wfengk]

I could build the Yelp querying functionality as a REST endpoint? that way you just need to make one additional call?

My problem with the crawler is that I wont get any more data after the 1000 restaurants (yelp cap) unless I search for one specifically