Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Core feature] Real-time Flyte Deck #5574

Open
2 of 8 tasks
pingsutw opened this issue Jul 21, 2024 · 5 comments
Open
2 of 8 tasks

[Core feature] Real-time Flyte Deck #5574

pingsutw opened this issue Jul 21, 2024 · 5 comments
Assignees
@MortalHappiness @Future-Outlier
Labels
enhancement New feature or request flytekit FlyteKit Python related issue flytepropeller ui Admin console user interface UX Issues that require UX-Design

Comments

@pingsutw
Copy link
Member

pingsutw commented Jul 21, 2024 •
edited by Future-Outlier
Loading

Motivation: Why do you think this is important?

It's extremely useful to see the metrics, data, logs, and links on the Flyte Deck while the task is running. However, currently, the Flyte Deck HTML is uploaded to the remote bucket only after the task succeeds.

image

Goal: What should the final outcome look like, ideally?

FlytePropeller

Flytekit

  • Upload a dummy deck when the task starts running.
  • Add publish_decks API, so users can decide when to upload the decks.
  • Upload flyte deck if the task fails.
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Flytekit Status</title>
    <style>
        body {
            display: flex;
            justify-content: center;
            align-items: center;
            height: 100vh;
            margin: 0;
            font-family: Arial, sans-serif;
            background-color: #f0f0f0;
        }
        .message {
            background-color: #ffffff;
            padding: 20px;
            border: 1px solid #cccccc;
            border-radius: 5px;
            box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
            text-align: center;
        }
    </style>
</head>
<body>
    <div class="message">
        <p> Flyte decks have not been created yet. </p>
    </div>
</body>
</html>
@task(enable_deck=True)
def foo():
    ctx = flytekit.current_context()
    ctx.publish_decks()

    for epoch in range(100):
       ...
       flytekit.Deck("metrics", ...)
       ctx.publish_decks()

FlyteConsole

  • Show the Flyte deck button while the task is running
  • Auto refresh flyte deck. Ifame should read the latest deck html in the remote bucket.

https://github.com/flyteorg/flyteconsole/blob/7636e32d2b1dbefb7d08fb79e5ed063c3ad44ca4/packages/oss-console/src/components/Executions/ExecutionDetails/ExecutionDetailsActions/FlyteDeckButton.tsx#L61

Describe alternatives you've considered

NA

Propose: Link/Inline OR Additional context

NA

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@pingsutw pingsutw added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers labels Jul 21, 2024
@runllm RunLLM
Copy link

runllm bot commented Jul 21, 2024

Currently, Flyte does not support real-time updates to Flyte Decks while a task is running. Decks are only uploaded to blob storage after the task completes. This limitation is discussed in this GitHub issue and this community discussion.

Would you like more detail on the current limitations and potential workarounds?

Sources:

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

@MortalHappiness
Copy link
Member

#take

@pingsutw pingsutw added flytekit FlyteKit Python related issue flytepropeller ui Admin console user interface UX Issues that require UX-Design and removed untriaged This issues has not yet been looked at by the Maintainers labels Jul 21, 2024
@eapolinario eapolinario moved this from Backlog to Assigned in Flyte Issues/PRs maintenance Jul 25, 2024
@Future-Outlier Future-Outlier self-assigned this Sep 3, 2024
@Future-Outlier
Copy link
Member

Suggestions by @eapolinario

hard parts:

  1. error handling (avoid crashing when rate limit happens)

This was referenced Oct 1, 2024
Open
Closed
@Future-Outlier
Copy link
Member

Related PR by @Yicheng-Lu-llll

flyteorg/flyteadmin#579

1 similar comment
@Future-Outlier
Copy link
Member

Related PR by @Yicheng-Lu-llll

flyteorg/flyteadmin#579

This was referenced Nov 27, 2024
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MortalHappiness MortalHappiness

@Future-Outlier Future-Outlier

Labels
enhancement New feature or request flytekit FlyteKit Python related issue flytepropeller ui Admin console user interface UX Issues that require UX-Design
Projects
Flyte Issues/PRs maintenance
Status: Assigned
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pingsutw @MortalHappiness @Future-Outlier