Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(robot-server): Flatten FastAPI routers #17169

Open
wants to merge 11 commits into
base: edge
Choose a base branch
from

Conversation

SyntaxColoring
Copy link
Contributor

@SyntaxColoring SyntaxColoring commented Dec 21, 2024

Overview

The Pydantic v1->v2 update makes server startup much slower. It turns out that a surprisingly large portion of that problem is due to inefficiencies in FastAPI's .include_router() method. This works around that.

Closes EXEC-1067.

Performance testing

Pydantic v1* Pydantic v2 before this PR Pydantic v2 with this PR
import robot_server.app on my laptop ~2.3s (flame graph) ~6.0s (flame graph) ~3.5s (flame graph)
import robot_server.app on a robot (estimated) 3-ish minutes (estimated) 2-ish minutes (estimated)
robot-server's entire lint+test CI workflow ~13 min ~21 min ~15 min

*Note that the comparison against Pydantic v1 is a bit apples-to-oranges, because Pydantic v2 does more things during import time so it can be faster after import time.

Raw flame graph files here.

Correctness testing

  • Make sure the integration tests keep keep passing.
  • Skim OpenAPI docs and make sure it doesn't look like anything has changed.

I could also add more focused unit tests for this if we think it's worth the time.

Changelog

Our constraints are basically:

  • On one hand, we want to call FastAPI's .include_router() as little as possible, for performance.
  • On the other, we do not want to reorganize our whole codebase around this. We want to keep organizing our routers as a tree, and we probably want everything to remain basically API-compatible with normal FastAPI. For instance, some future version of FastAPI might fix the underlying problem, so it would be good if we could easily undo all of this.

Therefore:

  • In server_utils, build a drop-in replacement for FastAPI's APIRouter. It is basically a subset of APIRouter's interface and functionality. There is some "cleverness" here to avoid having to duplicate the giant method signatures of .get(), .post(), etc.
  • In robot_server, switch as much as we can over to the drop-in. This is basically just a find+replace job, but there are a couple of small gotchas.

Review requests

  • Is there a cleaner alternative to my typing shenanigans? Should we just duplicate the method signatures from FastAPI?
  • Does the code explain itself well enough? Any naming or documentation feedback?
  • Is the performance improvement worth the complexity and weirdness? I've spent probably too long on this, but I wouldn't take it personally if we wanted to drop it and just eat the performance hit.

Risk assessment

Medium.

If something goes wrong here, I'd bet it would be in the part where we merge kwargs from different levels of the router tree. We want to match FastAPI behavior there, which seems like it can be pretty nuanced. Also, it is just some ugly dict manipulation.

This excludes service/session/router.py because it's using `APIRouter.url_path_for()`. That looks like something we can remove, but I'm not attempting it here.
These routers have a custom 422 response, and modify the OpenAPI spec accordingly at the router level, and FastBuildRouter isn't smart enough to know what to do with that right now.
Specifically, doing the equivalent of e.g. FastAPI.get(app, *args, **kwargs) instead of app.get(*args, **kwargs) was making the mocking pretty confusing.
@SyntaxColoring SyntaxColoring requested review from a team December 21, 2024 05:59
@SyntaxColoring SyntaxColoring marked this pull request as ready for review December 21, 2024 06:55
@SyntaxColoring SyntaxColoring requested a review from a team as a code owner December 21, 2024 06:55
@SyntaxColoring SyntaxColoring changed the title perf(robot-server): Flatten fastapi routers perf(robot-server): Flatten FastAPI routers Dec 21, 2024
Copy link
Contributor

@TamarZanzouri TamarZanzouri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this approach is clear. The only rename I would change is the _SomethingCallableLike to _CallableLike but no biggy. Thank you for fixing this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants