[Doc] Update build-from-source instructions (#3345)

MasterJH5574 · web-flow · commit 317b9943e0f6 · 2025-09-17T11:31:20.000-04:00
This PR updates the build-from-source instructions of MLC and TVM.
diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md
@@ -31,11 +31,11 @@ Steps to reproduce the behavior:
  - Operating system (e.g. Ubuntu/Windows/MacOS/...):
  - Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...)
  - How you installed MLC-LLM (`conda`, source):
- - How you installed TVM-Unity (`pip`, source):
+ - How you installed TVM (`pip`, source):
  - Python version (e.g. 3.10):
  - GPU driver version (if applicable):
  - CUDA/cuDNN version (if applicable):
- - TVM Unity Hash Tag (`python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))"`, applicable if you compile models):
+ - TVM Hash Tag (`python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))"`, applicable if you compile models):
  - Any other relevant information:
 
 ## Additional context
diff --git a/docs/community/faq.rst b/docs/community/faq.rst
@@ -12,5 +12,5 @@ This is a list of Frequently Asked Questions (FAQ) about the MLC-LLM. Feel free
    Please check our :doc:`/compilation/configure_quantization` tutorial.
 
 ... Why do I encounter an error ``free(): invalid pointer, Aborted (core dumped)`` at the end of model compilation?
-   This happens if you compiled TVM-Unity from source and didn't hide LLVM symbols in cmake configurations.
-   Please follow our instructions in :ref:`Building TVM Unity from Source  <tvm-unity-build-from-source>` tutorial to compile TVM-Unity which hides LLVM symbols, or use our pre-built MLC-LLM :doc:`pip wheels <../install/mlc_llm>`.
+   This happens if you compiled TVM from source and didn't hide LLVM symbols in cmake configurations.
+   Please follow our instructions in :ref:`Building TVM from Source  <tvm-build-from-source>` tutorial to compile TVM which hides LLVM symbols, or use our pre-built MLC-LLM :doc:`pip wheels <../install/mlc_llm>`.
diff --git a/docs/compilation/compile_models.rst b/docs/compilation/compile_models.rst
@@ -29,7 +29,7 @@ or be used to explicit create model libraries.
 We compile ``RedPajama-INCITE-Chat-3B-v1`` with ``q4f16_1`` as an example for all platforms.
 
 .. note::
-    Before you proceed, make sure you followed :ref:`install-tvm-unity`, a required
+    Before you proceed, make sure you followed :ref:`install-tvm`, a required
     backend to compile models with MLC LLM.
 
     Please also follow the instructions in :ref:`deploy-cli` / :ref:`deploy-python-engine` to obtain
@@ -40,7 +40,7 @@ We compile ``RedPajama-INCITE-Chat-3B-v1`` with ``q4f16_1`` as an example for al
     :depth: 1
     :local:
 
-0. Verify Installation
+1. Verify Installation
 ----------------------
 
 **Step 1. Verify mlc_llm**
@@ -60,13 +60,13 @@ installing the prebuilt package. Verify ``mlc_llm`` installation in command line
 
 **Step 2. Verify TVM**
 
-To compile models, you also need to follow :ref:`install-tvm-unity`.
-Here we verify ``tvm`` quickly with command line (for full verification, see :ref:`tvm-unity-validate`):
+To compile models, you also need to follow :ref:`install-tvm`.
+Here we verify ``tvm`` quickly with command line (for full verification, see :ref:`tvm-validate`):
 
 .. code:: bash
 
     $ python -c "import tvm; print(tvm.__file__)"
-    /some-path/lib/python3.11/site-packages/tvm/__init__.py
+    /some-path/lib/python3.13/site-packages/tvm/__init__.py
 
 1. Clone from HF and convert_weight
 -----------------------------------
diff --git a/docs/compilation/convert_weights.rst b/docs/compilation/convert_weights.rst
@@ -17,7 +17,7 @@ This can be extended to, e.g.:
 - Add ``Llama-2-uncensored`` when MLC already supports Llama-2
 
 .. note::
-    Before you proceed, make sure you followed :ref:`install-tvm-unity`, a required
+    Before you proceed, make sure you followed :ref:`install-tvm`, a required
     backend to compile models with MLC LLM.
 
     Please also follow the instructions in :ref:`deploy-cli` / :ref:`deploy-python-engine` to obtain
@@ -30,7 +30,7 @@ This can be extended to, e.g.:
 
 .. _verify_installation_for_compile:
 
-0. Verify installation
+1. Verify installation
 ----------------------
 
 **Step 1. Verify mlc_llm**
@@ -50,13 +50,13 @@ installing the prebuilt package. Verify ``mlc_llm`` installation in command line
 
 **Step 2. Verify TVM**
 
-To compile models, you also need to follow :ref:`install-tvm-unity`.
-Here we verify ``tvm`` quickly with command line (for full verification, see :ref:`tvm-unity-validate`):
+To compile models, you also need to follow :ref:`install-tvm`.
+Here we verify ``tvm`` quickly with command line (for full verification, see :ref:`tvm-validate`):
 
 .. code:: bash
 
     $ python -c "import tvm; print(tvm.__file__)"
-    /some-path/lib/python3.11/site-packages/tvm/__init__.py
+    /some-path/lib/python3.13/site-packages/tvm/__init__.py
 
 
 1. Clone from HF and convert_weight
diff --git a/docs/compilation/package_libraries_and_weights.rst b/docs/compilation/package_libraries_and_weights.rst
@@ -18,7 +18,7 @@ Introduction
 ------------
 
 To use ``mlc_llm package``, we must clone the source code of `MLC LLM <https://github.com/mlc-ai/mlc-llm>`_
-and `install the MLC LLM and TVM Unity package <https://llm.mlc.ai/docs/install/mlc_llm.html#option-1-prebuilt-package>`_.
+and `install the MLC LLM and TVM package <https://llm.mlc.ai/docs/install/mlc_llm.html#option-1-prebuilt-package>`_.
 Depending on the app we build, there might be some other dependencies, which are described in
 corresponding :ref:`iOS <deploy-ios>` and :ref:`Android <deploy-android>` tutorials.
 
diff --git a/docs/deploy/android.rst b/docs/deploy/android.rst
@@ -39,7 +39,7 @@ Prerequisite
   ANDROID_NDK: %HOME%/AppData/Local/Android/Sdk/ndk/27.0.11718014
   TVM_NDK_CC: %ANDROID_NDK%/toolchains/llvm/prebuilt/windows-x86_64/bin/aarch64-linux-android24-clang
 
-**JDK**, such as OpenJDK >= 17, to compile Java bindings of TVM Unity runtime.
+**JDK**, such as OpenJDK >= 17, to compile Java bindings of TVM runtime.
 We strongly recommend setting the ``JAVA_HOME`` to the JDK bundled with Android Studio.
 e.g.
 ``export JAVA_HOME=/Applications/Android\ Studio.app/Contents/jbr/Contents/Home`` for macOS.
@@ -52,7 +52,7 @@ Set up the following environment variable:
 
 Please ensure that the JDK versions for Android Studio and JAVA_HOME are the same.
 
-**TVM Unity runtime** is placed under `3rdparty/tvm <https://github.com/mlc-ai/mlc-llm/tree/main/3rdparty>`__ in MLC LLM, so there is no need to install anything extra. Set up the following environment variable:
+**TVM runtime** is placed under `3rdparty/tvm <https://github.com/mlc-ai/mlc-llm/tree/main/3rdparty>`__ in MLC LLM, so there is no need to install anything extra. Set up the following environment variable:
 
 - ``export TVM_SOURCE_DIR=/path/to/mlc-llm/3rdparty/tvm``.
 
@@ -71,7 +71,7 @@ Check if **environment variable** are properly set as the last check. One way to
   export ANDROID_NDK=...  # Android NDK toolchain
   export TVM_NDK_CC=...   # Android NDK clang
   export JAVA_HOME=...    # Java
-  export TVM_SOURCE_DIR=...     # TVM Unity runtime
+  export TVM_SOURCE_DIR=...     # TVM runtime
 
 Additional Guides for Windows Users
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/docs/deploy/ios.rst b/docs/deploy/ios.rst
@@ -88,11 +88,11 @@ Please make sure ``dist/`` follows the structure below, except the optional mode
    │   ├── mlc-app-config.json  # The app config JSON file.
    │   └── [optional model weights]
    └── lib
-      ├── libmlc_llm.a          # A lightweight interface to interact with LLM, tokenizer, and TVM Unity runtime.
+      ├── libmlc_llm.a          # A lightweight interface to interact with LLM, tokenizer, and TVM runtime.
       ├── libmodel_iphone.a     # The compiled model lib.
       ├── libsentencepiece.a    # SentencePiece tokenizer
       ├── libtokenizers_cpp.a   # Huggingface tokenizer.
-      └── libtvm_runtime.a      # TVM Unity runtime.
+      └── libtvm_runtime.a      # TVM runtime.
 
 
 .. note::
diff --git a/docs/deploy/webllm.rst b/docs/deploy/webllm.rst
@@ -106,13 +106,13 @@ installing the prebuilt package. Verify ``mlc_llm`` installation in command line
 
 **Step 2. Verify TVM**
 
-To compile models, you also need to follow :ref:`install-tvm-unity`.
-Here we verify ``tvm`` quickly with command line (for full verification, see :ref:`tvm-unity-validate`):
+To compile models, you also need to follow :ref:`install-tvm`.
+Here we verify ``tvm`` quickly with command line (for full verification, see :ref:`tvm-validate`):
 
 .. code:: bash
 
     $ python -c "import tvm; print(tvm.__file__)"
-    /some-path/lib/python3.11/site-packages/tvm/__init__.py
+    /some-path/lib/python3.13/site-packages/tvm/__init__.py
 
 
 .. _webllm-add-model-variant:
diff --git a/docs/install/emcc.rst b/docs/install/emcc.rst
@@ -37,7 +37,7 @@ Step 2: Set TVM_SOURCE_DIR and MLC_LLM_SOURCE_DIR
 -------------------------------------------------
 
 We need to set a path to a tvm source in order to build tvm runtime.
-Note that you do not need to build tvm unity from the source. The source here is only used to build the web runtime component.
+Note that you do not need to build TVM from the source. The source here is only used to build the web runtime component.
 Set environment variable in your shell startup profile in to point to ``3rdparty/tvm`` (if preferred, you could also
 point to your own TVM address if you installed TVM from source).
 
diff --git a/docs/install/gpu.rst b/docs/install/gpu.rst
@@ -105,7 +105,7 @@ After installation, you can run ``vulkaninfo`` in command line and see if you ca
 Vulkan SDK
 ----------
 
-Vulkan SDK is required for compiling models to Vulkan backend. To build TVM Unity compiler from source, you will need to install Vulkan SDK as a dependency, but our :doc:`pre-built wheels <../install/mlc_llm>` already ships with Vulkan SDK.
+Vulkan SDK is required for compiling models to Vulkan backend. To build TVM compiler from source, you will need to install Vulkan SDK as a dependency, but our :doc:`pre-built wheels <../install/mlc_llm>` already ships with Vulkan SDK.
 
 Check Vulkan SDK installation guide according to your platform:
 
@@ -127,7 +127,7 @@ Check Vulkan SDK installation guide according to your platform:
 
       `Getting Started with the macOS Vulkan SDK <https://vulkan.lunarg.com/doc/sdk/latest/mac/getting_started.html>`__
 
-Please refer to installation and setup page for next steps to build TVM-Unity from source.
+Please refer to installation and setup page for next steps to build TVM from source.
 
 OpenCL SDK
 ----------
diff --git a/docs/install/mlc_llm.rst b/docs/install/mlc_llm.rst
@@ -82,12 +82,12 @@ Select your operating system/compute platform and run the command in your termin
 
                 conda install -c conda-forge libstdcxx-ng
 
-            Besides, we would recommend using Python 3.11; so if you are creating a new environment,
+            Besides, we would recommend using Python 3.13; so if you are creating a new environment,
             you could use the following command:
 
             .. code-block:: bash
 
-                conda create --name mlc-prebuilt  python=3.11
+                conda create --name mlc-prebuilt  python=3.13
 
     .. tab:: macOS
 
@@ -154,7 +154,7 @@ Then you can verify installation in command line:
 .. code-block:: bash
 
     python -c "import mlc_llm; print(mlc_llm)"
-    # Prints out: <module 'mlc_llm' from '/path-to-env/lib/python3.11/site-packages/mlc_llm/__init__.py'>
+    # Prints out: <module 'mlc_llm' from '/path-to-env/lib/python3.13/site-packages/mlc_llm/__init__.py'>
 
 |
 
@@ -188,13 +188,13 @@ This step is useful when you want to make modification or obtain a specific vers
         "cmake>=3.24" \
         rust \
         git \
-        python=3.11
+        python=3.13
     # enter the build environment
     conda activate mlc-chat-venv
 
 .. note::
-    For runtime, :doc:`TVM Unity </install/tvm>` compiler is not a dependency for MLCChat CLI or Python API. Only TVM's runtime is required, which is automatically included in `3rdparty/tvm <https://github.com/mlc-ai/mlc-llm/tree/main/3rdparty>`_.
-    However, if you would like to compile your own models, you need to follow :doc:`TVM Unity </install/tvm>`.
+    For runtime, :doc:`TVM </install/tvm>` compiler is not a dependency for MLCChat CLI or Python API. Only TVM's runtime is required, which is automatically included in `3rdparty/tvm <https://github.com/mlc-ai/mlc-llm/tree/main/3rdparty>`_.
+    However, if you would like to compile your own models, you need to follow :doc:`TVM </install/tvm>`.
 
 **Step 2. Configure and build.** A standard git-based workflow is recommended to download MLC LLM, after which you can specify build requirements with our lightweight config generation tool:
 
@@ -208,7 +208,7 @@ This step is useful when you want to make modification or obtain a specific vers
     # generate build configuration
     python ../cmake/gen_cmake_config.py
     # build mlc_llm libraries
-    cmake .. && cmake --build . --parallel $(nproc) && cd ..
+    cmake .. && make -j $(nproc) && cd ..
 
 **Step 3. Install via Python.** We recommend that you install ``mlc_llm`` as a Python package, giving you
 access to ``mlc_llm.compile``, ``mlc_llm.MLCEngine``, and the CLI.
diff --git a/docs/install/tvm.rst b/docs/install/tvm.rst
@@ -1,6 +1,6 @@
-.. _install-tvm-unity:
+.. _install-tvm:
 
-Install TVM Unity Compiler
+Install TVM Compiler
 ==========================
 
 .. contents:: Table of Contents
@@ -14,14 +14,14 @@ Install TVM Unity Compiler
 - Supporting both inference and training;
 - Productive python-first compiler implementation. As a concrete example, MLC LLM compilation is implemented in pure python using its API.
 
-TVM Unity can be installed directly from a prebuilt developer package, or built from source.
+TVM can be installed directly from a prebuilt developer package, or built from source.
 
-.. _tvm-unity-prebuilt-package:
+.. _tvm-prebuilt-package:
 
 Option 1. Prebuilt Package
 --------------------------
 
-A nightly prebuilt Python package of Apache TVM Unity is provided.
+A nightly prebuilt Python package of Apache TVM is provided.
 
 .. note::
     ❗ Whenever using Python, it is highly recommended to use **conda** to manage an isolated Python environment to avoid missing dependencies, incompatible versions, and package conflicts.
@@ -131,12 +131,12 @@ A nightly prebuilt Python package of Apache TVM Unity is provided.
 
             conda install zstd
 
-.. _tvm-unity-build-from-source:
+.. _tvm-build-from-source:
 
 Option 2. Build from Source
 ---------------------------
 
-While it is generally recommended to always use the prebuilt TVM Unity, if you require more customization, you may need to build it from source. **NOTE.** this should only be attempted if you are familiar with the intricacies of C++, CMake, LLVM, Python, and other related systems.
+While it is generally recommended to always use the prebuilt TVM, if you require more customization, you may need to build it from source. **NOTE.** this should only be attempted if you are familiar with the intricacies of C++, CMake, LLVM, Python, and other related systems.
 
 .. collapse:: Details
 
@@ -169,25 +169,22 @@ While it is generally recommended to always use the prebuilt TVM Unity, if you r
             "llvmdev>=15" \
             "cmake>=3.24" \
             git \
-            python=3.11
+            python=3.13
         # enter the build environment
         conda activate tvm-build-venv
 
-    **Step 2. Configure and build.** Standard git-based workflow are recommended to download Apache TVM Unity, and then specify build requirements in ``config.cmake``:
+    **Step 2. Configure and build.** Standard git-based workflow are recommended to download Apache TVM, and then specify build requirements in ``config.cmake``:
 
     .. code-block:: bash
-        :caption: Download TVM Unity from GitHub
+        :caption: Download TVM from GitHub
 
         # clone from GitHub
-        git clone --recursive https://github.com/mlc-ai/relax.git tvm-unity && cd tvm-unity
+        git clone --recursive https://github.com/apache/tvm.git && cd tvm
         # create the build directory
         rm -rf build && mkdir build && cd build
         # specify build requirements in `config.cmake`
         cp ../cmake/config.cmake .
 
-    .. note::
-        We are temporarily using `mlc-ai/relax <https://github.com/mlc-ai/relax>`_ instead, which comes with several temporary outstanding changes that we will upstream to Apache TVM's `unity branch <https://github.com/apache/tvm/tree/unity>`_.
-
     We want to specifically tweak the following flags by appending them to the end of the configuration file:
 
     .. code-block:: bash
@@ -200,10 +197,21 @@ While it is generally recommended to always use the prebuilt TVM Unity, if you r
         echo "set(HIDE_PRIVATE_SYMBOLS ON)" >> config.cmake
         # GPU SDKs, turn on if needed
         echo "set(USE_CUDA   OFF)" >> config.cmake
+        echo "set(USE_ROCM   OFF)" >> config.cmake
         echo "set(USE_METAL  OFF)" >> config.cmake
         echo "set(USE_VULKAN OFF)" >> config.cmake
         echo "set(USE_OPENCL OFF)" >> config.cmake
+        # Below are options for CUDA, turn on if needed
+        # CUDA_ARCH is the cuda compute capability of your GPU.
+        # Examples: 89 for 4090, 90a for H100/H200, 100a for B200.
+        # Reference: https://developer.nvidia.com/cuda-gpus
         echo "set(CMAKE_CUDA_ARCHITECTURES YOUR_CUDA_COMPUTE_CAPABILITY_HERE)" >> config.cmake
+        echo "set(USE_CUBLAS ON)" >> config.cmake
+        echo "set(USE_CUTLASS ON)" >> config.cmake
+        echo "set(USE_THRUST ON)" >> config.cmake
+        echo "set(USE_NVTX ON)" >> config.cmake
+        # Below is the option for ROCM, turn on if needed
+        echo "set(USE_HIPBLAS ON)" >> config.cmake
 
     .. note::
         ``HIDE_PRIVATE_SYMBOLS`` is a configuration option that enables the ``-fvisibility=hidden`` flag. This flag helps prevent potential symbol conflicts between TVM and PyTorch. These conflicts arise due to the frameworks shipping LLVMs of different versions.
@@ -219,50 +227,50 @@ While it is generally recommended to always use the prebuilt TVM Unity, if you r
     .. code-block:: bash
         :caption: Build ``libtvm`` using cmake and cmake
 
-        cmake .. && cmake --build . --parallel $(nproc)
+        cmake .. && make -j $(nproc) && cd ..
 
-    A success build should produce ``libtvm`` and ``libtvm_runtime`` under ``/path-tvm-unity/build/`` directory.
+    A success build should produce ``libtvm`` and ``libtvm_runtime`` under ``/path-tvm/build/`` directory.
 
     Leaving the build environment ``tvm-build-venv``, there are two ways to install the successful build into your environment:
 
     .. tabs ::
 
        .. code-tab :: bash Install via environment variable
 
-          export PYTHONPATH=/path-to-tvm-unity/python:$PYTHONPATH
+          export PYTHONPATH=/path-to-tvm/python:$PYTHONPATH
 
        .. code-tab :: bash Install via pip local project
 
           conda activate your-own-env
           conda install python # make sure python is installed
-          cd /path-to-tvm-unity/python
+          cd /path-to-tvm/python
           pip install -e .
 
 .. `|` adds a blank line
 
 |
 
-.. _tvm-unity-validate:
+.. _tvm-validate:
 
 Validate TVM Installation
 -------------------------
 
 Using a compiler infrastructure with multiple language bindings could be error-prone.
-Therefore, it is highly recommended to validate TVM Unity installation before use.
+Therefore, it is highly recommended to validate TVM installation before use.
 
 **Step 1. Locate TVM Python package.** The following command can help confirm that TVM is properly installed as a python package and provide the location of the TVM python package:
 
 .. code-block:: bash
 
     >>> python -c "import tvm; print(tvm.__file__)"
-    /some-path/lib/python3.11/site-packages/tvm/__init__.py
+    /some-path/lib/python3.13/site-packages/tvm/__init__.py
 
 **Step 2. Confirm which TVM library is used.** When maintaining multiple build or installation of TVM, it becomes important to double check if the python package is using the proper ``libtvm`` with the following command:
 
 .. code-block:: bash
 
     >>> python -c "import tvm; print(tvm.base._LIB)"
-    <CDLL '/some-path/lib/python3.11/site-packages/tvm/libtvm.dylib', handle 95ada510 at 0x1030e4e50>
+    <CDLL '/some-path/lib/python3.13/site-packages/tvm/libtvm.dylib', handle 95ada510 at 0x1030e4e50>
 
 **Step 3. Reflect TVM build option.** Sometimes when downstream application fails, it could likely be some mistakes with a wrong TVM commit, or wrong build flags. To find it out, the following commands will be helpful:
 
diff --git a/python/mlc_llm/interface/compile.py b/python/mlc_llm/interface/compile.py
