# ONNX Runtime Eager Mode Support for PyTorch

## Build and test
1. sudo apt-get install python3-dev
2. Upgrade CMAKE to > version 3.18.1.
3. Create and activate a virtual environment:
		a. python3 -m venv ~/venvs/onnxruntime
		b. source ~/venvs/onnxruntime/bin/activate
4. pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu
5. pip3 install parameterized
6. pip install packaging wheel
7. Sudo apt-get install mpich libmpich-dev libcomp-11-dev (or use '--use_mpi=False' in the build command)
8. Sudo apt-get update (optional if you think software is out of date and it might show out dated stuff to remove)
9. Clone microsoft/onnxruntime

To build, go to the root of the onnxruntime repository and run:
./build.sh --cmake_generator Ninja --config Debug --build_shared_lib --parallel --build_eager_mode --enable_training --enable_pybind  --build_wheel --skip_tests

To run the eager tests:
 PYTHONPATH=~/{onnxruntime repo}/build/Linux/Debug python3 ~/{onnxruntime repo}/orttraining/orttraining/eager/test/ort_ops.py

## Mapping aten cpp namespace functions to onnx ops
ORT Eager uses opgen to generate C++ shim code that intercepts PyTorch operator calls and executes them with the ORT
backend. The opgen script is executed as part of the build process, and it can be found in
`orttraining/orttraining/eager/opgen/opgen.py.` The shim code generated by opgen is responsible for mapping aten inputs
to the ORT values passed into the onnx ops, invoking the onnx op, and converting outputs to the correct return
type. Opgen also creates Python bindings for the C++ shim code to allow it to be called directly from PyTorch when
using the ORT eager backend.

Useful links
- [Onnx Op Schema](https://github.com/onnx/onnx/blob/main/docs/Operators.md)
- [Aten native ops](https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/native_functions.yaml)

For mapping existing aten ops to onnx files, start in `orttraining/orttraining/eager/opgen/opgen/atenops.py.` This file
drives the generator which the build runs and produces `orttraining/orttraining/eager/ort_aten.g.cpp`. Looking at the
generated code will be helpful to understand the aten op signature and how onnx ops are invoked.


In the simple case, add the aten op and the mapped onnx op to the `hand_implemented` list. As
an example `"aten::t": Transpose("self"),` maps the aten transpose function `t` to the onnx op `Transpose` and maps the
`self` input param of `t` to the first argument of `Transpose`.


Often mapping aten to onnx is not one to one, so composing is supported. Example
`"aten::zeros_like": ConstantOfShape(Shape("self")),`.


In some case more data shaping is required, so only the signature should be created such as `"aten::equal": SignatureOnly(),`.
The implementation is then added to `orttraining/orttraining/eager/ort_aten.cpp`


It may also be necessary to expose an onnx op in eager mode that does not exist as a native aten op. For this scenario,
start in `orttraining/orttraining/eager/opgen/opgen/custom_ops.py`, which will generate code found in
`orttraining/orttraining/eager/ort_customops.g.cpp`. Custom ops can then be invoked from PyTorch eager like
`torch.ops.ort.<op_name>`. Note that opgen for custom ops will also need header/schema declarations, found in
`orttraining/orttraining/eager/opgen/CustomOpDeclarations.h` (equivalent to RegistrationDeclarations.h for aten ops).


Please add tests for all ops. Tests are defined in `orttraining/orttraining/eager/test/ort_ops.py`

## Decisions worth noting
- Resizing the output tensor: Aten supports resizing any out tensor but prints a warning this is depracated and support
will end. With that in mind, we have decided to error in `resize_output` if the output tensor is not empty or already
the right shape.
- Python bindings for ONNX ops: opgen will use the TORCH_LIBRARY binding API instead of PyBind11 to bind ATen and custom
ops to generated C++ code. Using TORCH_LIBRARY and TORCH_LIBRARY_IMPL allows us to introduce backend-specific bindings
for native and custom ops while using the familiar PyTorch interface. More information:
[TORCH_LIBRARY](https://pytorch.org/tutorials/advanced/torch_script_custom_ops.html),
[TORCH_LIBRARY_IMPL](https://pytorch.org/tutorials/advanced/extend_dispatcher.html).
