Fix WOQ int8 failures

[leslie-fang-intel]

Fix WOQ int8 failures as discussed in #843 (comment)

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/884

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 828571c with merge base 8236a87 ():

NEW FAILURE - The following job has failed:

• Run Regression Tests / test (CPU Nightly, linux.4xlarge, --pre torch --index-url https://download.pytorch.org/whl/nightl... / linux-job (gh)
  test/integration/test_integration.py::TestSubclass::test_int8_weight_only_quant_subclass_api_1_cpu

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Run Regression Tests / test (CUDA Nightly, linux.g5.12xlarge.nvidia.gpu, --pre torch --index-url https://download.pytorc... / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[leslie-fang-intel]

cc @jerryzh168 @bdhirsh

[jerryzh168]

when is freezing set? how do user know that they need to call unwrap_tensor_subclass in that case?

[leslie-fang-intel]

Hi Jerry, freezing is setting in

ao/test/integration/test_integration.py

Line 829 in 8236a87

@torch._inductor.config.patch({"freezing": True})

when running the UT. User can also set it as torch._inductor.config.freezing=True in script or using the environment variable as TORCHINDUCTOR_FREEZING=1

[leslie-fang-intel]

how do user know that they need to call unwrap_tensor_subclass in that case?

That's a good question. Did you mean we should write document somewhere? And the root-cause for why freezing needs unwrap_tensor_subclass is in pytorch/pytorch#123522.

[jerryzh168]

I see, thanks for the context! yeah I think it would be good to add this in the README: https://github.com/pytorch/ao/tree/main/torchao/quantization#workaround-with-unwrap_tensor_subclass-for-export-aoti-and-torchcompile-pytorch-24-and-before-only and link the issue as well

[leslie-fang-intel]

Thanks and adding the readme with issue link in this PR. please help to take a look again.

[jerryzh168]

LGTM

[jerryzh168]

please fix CI before landing

[leslie-fang-intel]

please fix CI before landing

Thanks @jerryzh168, we should land the PR in PyTorch at first.
Could you help to take a review pytorch/pytorch#135928?

[leslie-fang-intel]

Hi @jerryzh168, since pytorch/pytorch#135928 has been merged into PyTorch, could you approve the CI run for this PR to check if it passes all tests?

[leslie-fang-intel]

It seems test_int8_weight_only_quant_subclass_api with float16 inputs still failed which is probably due to numerical issue. Will take a further look.