Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[aDAG][Core] Weird exception if ref is deleted after adag teardown #47687

Open
rkooo567 opened this issue Sep 16, 2024 · 1 comment · May be fixed by #47689
Open

[aDAG][Core] Weird exception if ref is deleted after adag teardown #47687

rkooo567 opened this issue Sep 16, 2024 · 1 comment · May be fixed by #47689
Labels
accelerated-dag bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core triage Needs triage (eg: priority, bug/not-bug, and owning component)

Comments

@rkooo567
Copy link
Contributor

What happened + What you expected to happen

We call ray.get on ref del, but if the dag is already teardown, it raises an weird exception

Versions / Dependencies

master

Reproduction script

import ray
from ray.dag import InputNode, MultiOutputNode

ray.init()

@ray.remote
class A:
    def f(self, inp):
        return 1

a = A.remote()
b = A.remote()
with InputNode() as inp:
    x = b.f.bind(inp)
    y = a.f.bind(inp)
    dag = MultiOutputNode([x,y ])
adag = dag.experimental_compile()
refs = adag.execute(1)
adag.teardown()

Issue Severity

None

@rkooo567 rkooo567 added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) accelerated-dag labels Sep 16, 2024
@anyscalesam anyscalesam added the core Issues that should be addressed in Ray Core label Sep 16, 2024
@rkooo567 rkooo567 linked a pull request Sep 16, 2024 that will close this issue
8 tasks
@ruisearch42
Copy link
Contributor

Can you post the stack trace of the exception?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accelerated-dag bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core triage Needs triage (eg: priority, bug/not-bug, and owning component)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants