Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tasks are uptodate even though their task_dependency is not #445

Open
wassupxP opened this issue Dec 15, 2022 · 6 comments
Open

tasks are uptodate even though their task_dependency is not #445

wassupxP opened this issue Dec 15, 2022 · 6 comments
Labels
Milestone

Comments

@wassupxP
Copy link

wassupxP commented Dec 15, 2022

Issue:
After version bump I found out, that tasks are uptodate even though some of their task_deps are not. Only subtask is rebuilt afterwards and not whole tree to the parent

      task1:up-to-date
      /          \
  task2:run      task3:up-to-date

Not sure whether this was intentional or really a bugfix.

Workaround:
In task_loader inject True or False to "uptodate" for each task in task_deps, depending on status of dependent task.

# Get current status of dependent task
result = doit_dependency_manager.get_status(task_object, task_dict)
# Add result to list as a boolean
uptodate.append(True if result.status == "up-to-date" else False)

Environment

  1. OS: win
  2. python version: 3.9
  3. doit version: 0.35.0

Let me know if additional informations or sth is needed.

Fund with Polar
@schettino72
Copy link
Member

Please include a minimal dodo.py that reproduces the problem.

@wassupxP
Copy link
Author

wassupxP commented Jan 16, 2023

Hi, it is a bit nasty, but you should be able to reproduce the issue from it. Run on doit 0.35.0

import os

def task_parent():
    curr_file = os.path.dirname(os.path.abspath(__file__))
    def action():
        with open(os.path.join(curr_file, "file.txt"), "w+") as f:
            f.write("MOCK")

    return {'actions': [(action,)],
            'uptodate': [os.path.isfile(os.path.join(curr_file, "file.txt"))],
            'basename': 'parent',
            'task_dep': ["child"]
            }

def task_child2():
    curr_file = os.path.dirname(os.path.abspath(__file__))
    def action():
        with open(os.path.join(curr_file, "file2.txt"), "w+") as f:
            f.write("MOCK")

    return {'actions': [(action,)],
            'uptodate': [os.path.isfile(os.path.join(curr_file, "file2.txt"))],
            'basename': 'child',
            }

Including also console log

$ ls
__pycache__/  dodo.py  share/
$ python.exe -m doit parent
.  child
.  parent
$ ls
__pycache__/  dodo.py  file.txt  file2.txt  share/
$ rm file2.txt
$ python.exe -m doit info parent

parent

status     : up-to-date

task_dep   :
 - child
$ python.exe -m doit info child

child

status     : run
 * The following uptodate objects evaluate to false:
    - False (args=None, kwargs=None)


Please let me know if you need additional info.

@schettino72
Copy link
Member

If you execute again the output would be:

.  child
-  parent

So the parent is really up-to-date.

A task-dependency only indicates that another task should be “executed” before itself. [1]
I don't know how to make this more clear in the docs... I will add a FAQ because it is a common misconception.

[1] https://pydoit.org/dependencies.html#task-dependency

@schettino72 schettino72 added this to the 0.37 milestone Jan 16, 2023
@ugomancz
Copy link

@schettino72 Can I ask why the up-to-date state isn't propagated? Have you considered adding this feature? Since it's a common "misconception", it's probably a behavior that is illogical and unexpected by most users who encounter it.

@wassupxP
Copy link
Author

If you execute again the output would be:

.  child
-  parent

So the parent is really up-to-date.

A task-dependency only indicates that another task should be “executed” before itself. [1] I don't know how to make this more clear in the docs... I will add a FAQ because it is a common misconception.

[1] https://pydoit.org/dependencies.html#task-dependency

Thanks for info. Well this behavior is easily overridable, just wanted to report it. But from my point of view, if task depends on another task, its final state should be also dependent on state of child task. Thanks for explanation and sorry for wasting your time. FAQ section will surely help plenty of other users! :)

@schettino72
Copy link
Member

But from my point of view, if task depends on another task, its final state should be also dependent on state of child task.

Well, that's your use case.
The system was designed so its core concepts are orthogonal. This gives more flexibility in the long run.

You could repeat the same "file_dep" in the parent task.
Or you probable want to use a result_dep [1]

[1] https://pydoit.org/uptodate.html?highlight=result_dep#result-dependency

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants