docs: Adding language recommendation

[franciscojavierarceo]

What this PR does / why we need it:

This PR adds documentation about why we recommend using Feast with a Python Microservice.

Which issue(s) this PR fixes:

#4266

Fixes

#4266

[tokoko]

Here we go 😄 I'm obviously very conflicted about this. I think we might be conflating two points here:

• Should I reimplement feature computation logic? - tbh, I think this is the part this text is addressing rather than serving language choice. I'm very much on board here, you shouldn't reimplement because of skew, effort and all the other reasons outlined in the doc. But if you manage to stick to Precomputation is the way religiously (i.e. you're not using odfvs), that also means that you have effectively decoupled computation from serving, so the discussion regarding reimplementation of features has no bearing to serving environment language choice anymore.

• Should I run a feature server in something other than python? - If we disregard odfvs for a minute, this is more of an implementation/maintenance burden for us rather than a user. Of course, I'd also go with python at this point in time because I'm not sure others work at all, but if we make them work and if we reimplement online store retrieval logic for most of online store implementations in java/go, there's really no reason why someone shouldn't switch to them (other than odfvs).

  Executing ODFVs written in python in a non-python serving environment is admittedly very tricky, but also something that's on us to solve, not on the user. If we manage to find good ways to do it w/o requiring reimplementation (by optimizing transformation server, making arrow-based go-python interop work or by somehow leveraging substrait), that also becomes a non-issue for the user.

[franciscojavierarceo]

So in my last role I used ODFVs to create features and then persisted that output to a regular FV.

This a simple hack to allow us to launch what we needed faster but the real solution is to enable FVs to have feature computations equivalent to how ODFVs work with the Python mode.

The point is that feature transformations (i.e., precomputation) should happen in Python as well. ODFV will still need to support some light weight calculations (e.g., date differences from datetime.now()).

Let me in how what you think and thanks for the feedback! I can definitely incorporate this feedback in.

[tokoko]

You're referring to BatchFeatureView concept, right? (in tecton lingo, we still have to implement an alternative) I think I agree there, When implemented, BatchFeatureViews should only support python as far as feast is concerned. Having said that, we should also keep supporting externally computed features with normal FeatureView objects (Feature Table in tecton lingo). Those externally computed features might or might not be written in python, of course, we don't really have any say over there. I don't think we disagree there, just wanted to stress that none of this (except odfvs) has any bearing to the serving environment.

[franciscojavierarceo]

yeah agreed with that. I'll clarify this in the doc more and I'll cut a PR to support Feature Transformations in regular feature views

[EXPEbdodla]

The problem I see with this recommendation is online feature retrieval latency. Python has high latencies compared to Go or Java option. Do you think its better to mention the latency impact?

[franciscojavierarceo]

I tried to address this point in this statement:

Precomputing features is the recommended optimal path to ensure low latency performance. Reducing feature serving to a lightweight database lookup is the ideal pattern, which means the marginal overhead of Python should be tolerable.

But I can be more explicit.

[tokoko]

That definitely should be a factor. Even if you precompute, there will be applications out there with low-latency requirements and high enough load for which python server performance itself might become a bottleneck. I guess we are sort of trying to address that with introducing asyncio in python online retrieval, but even that might not be enough for some use cases.

[franciscojavierarceo]

While that is true in theory, in practice Python works very well at quite high scale so my goal is to make it clear that we recommend Python.

Regardless, I see that this is in the overview section so I'll add this snippet in it.