Skip to content

API Call tracking - #348

Open
mcoughlin wants to merge 5 commits into
cesium-ml:mainfrom
mcoughlin:api_tracking
Open

mcoughlin wants to merge 5 commits into
cesium-ml:mainfrom
mcoughlin:api_tracking

Conversation

@mcoughlin

Copy link
Copy Markdown
Collaborator

This PR adds API Call tracking.

@profjsb

profjsb commented Jan 15, 2023

Copy link
Copy Markdown
Collaborator

Looks good. Question is whether we want to do this logging in the DB itself or another DB. I'm worried about quickly blowing up the DB in size. Can we get an estimate of expected DB growth with the current load if this goes in?

@stefanv

stefanv commented Jan 15, 2023

Copy link
Copy Markdown
Contributor

FWIW, all connections are logged in gcloud. This may still be the easiest route to getting the statistics you want, but worth seeing what they store.

@mcoughlin

Copy link
Copy Markdown
Collaborator Author

@profjsb Posting about 2000 API calls, I get a table size of 840 kB. So if we assume an API call a second on production, we would get 36 MB per day.

@guynir42 guynir42 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I have a couple of suggestions.

Comment thread app/handlers/base.py Outdated
api_call = APICall(
user_id=user_id,
method=self.request.method,
uri=self.request.uri.split("?")[0],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, but maybe worth while to save the parameters as well? Maybe limit the number of characters, just in case we get super-long parameter strings, but those parameters could contain lots of useful info.

Also, consider clipping the beginning of the string (I don't know if URI contains the full address or just the thing after api/)

Comment thread app/handlers/base.py
method=self.request.method,
uri=self.request.uri.split("?")[0],
size=sizeof(data),
success=False,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you stick something like self.start_time = time.time() up in the prepare method, you could also get the runtime of the query. That should work unless there's async stuff happening, in which case who knows what will happen... but it is worth trying out and comparing to timing data you get from the side calling the API.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants