Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration with tract #104

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Integration with tract #104

wants to merge 2 commits into from

Conversation

dsseng
Copy link

@dsseng dsseng commented May 6, 2022

Migrated code from sonos/tract#697, see discussion there for reasoning. The goal is to combine powers: leverage both model loading and optimization capabilities of tract-core and shader compilation/execution implementations from wonnx.

Tasks (partially from sonos/tract#697):

  • GPUTensor
    • Import
      • Proper type for imported tensor
    • Export
    • Intermediate data in GPU memory
    • Pass tensor strides as uniforms
    • Have way of processing rank 4 tensors
  • Ops are going to be used from existing shaders
  • Runner for models
    • Managing GPU memory
      • Free buffers no longer in use to allow for models larger than GPU memory
  • Examples working
    • tensorflow-mobilenet-v2
    • Others from tract repo
  • Test various platforms
    • Linux
      • Vulkan + RADV
      • Other GPUs
      • Various embedded systems like RPi
    • Windows
    • macOS and iOS
    • Android
    • WASM
      • WebGPU
      • WebGL

Ported GPUTensor handling code
Comment on lines +3 to +9
version = "0.0.1"
license = "MIT/Apache-2.0"
authors = ["Dmitry Sharshakov <[email protected]>"]
description = "An integration crate for using GPU compiler from wonnx to accelerate tract model inference"
repository = "https://github.com/snipsco/tract"
keywords = ["TensorFlow", "NeuralNetworks"]
categories = ["science"]
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be done properly for this repository.

@dsseng
Copy link
Author

dsseng commented May 7, 2022

@haixuanTao now I certainly need to have a chat or some other quicker way of getting in touch with you. We have to discuss many stuff about this PR

@haixuanTao
Copy link
Collaborator

@haixuanTao now I certainly need to have a chat or some other quicker way of getting in touch with you. We have to discuss many stuff about this PR

Sure. Let's find a schedule: https://calendly.com/tao-xavier/60min

@dsseng
Copy link
Author

dsseng commented May 13, 2022

Set up 2PM UTC+3. Are you comfortable talking via Discord or prefer texting?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants