Minutes from 77th OpenVDB TSC meeting, Jan 26th, 2021, (EDT)

Attendees: *Nick* A., *Jeff* L., *Ken* M., *Dan* B., *Andre* P.

Additional: Vojta Kuzel, JT Nelson (Blender), Laura Lediaev (Sony Pictures Imageworks), Bruce Cherniak (Intel), Johannes Meng (Intel). 

Regrets: Peter C.

1) Forum (DAG)
2) Half
3) VDB 8 bugfix progress #936
4) Blender update (JT)
5) Min boost requirement  #935
6) AX SOP progress #931
7) Comment on USD
8) Next Meeting

1) Question in the forum: why do we 

Octree v.s. VDB grid. Octree has a small fan-out factor (8) while the smallest fan-out factor in a VDB grid is 512. The chance of two leaf nodes having collision is small in VDB. 

An OpenVDB grid is also designed to change topology, whereas DAG is mainly designed to be a (preconditioning) data structure to be used for rendering. For this purpose, a NanoVDB grid is maybe more suitable for the problem. Updating a DAG dynamically will be difficult. However, Jeff Lait mentioned that NanoVDB also adds bounding box information in the Leaf Nodes, making it not suitable for a DAG-like data structure (at least in its current implementation). 

Ken thinks that the idea is interesting and encourages Vojta Kuzel to explore it. However, having DAG as a first class citizen in OpenVDB will be difficult because we allow the tree to dynamically change. Ken mentions that the reason people shy away from implementing it is because of skepticism over whether it will work out in practice. 

Dan mentions that applying this idea to a Mask Grid may be more appropriate. 

2) Half

Dan mentions that Carry from ILM reached out. Dan thinks that Carry would like an ASWF project to use another ASWF project so we can give feedback. The look up table in OpenEXR half has gone away in version 2.5. Dan also mentions using Imath’s vector classes. Ken says that the vector classes discussion is a bigger discussion that should be discussed at another point. 

Jeff mentions that we want to reduce OpenVDB’s dependency and that no matter what we do, we need to put the embedded half implementation in a new namespace. Ken would like to have this PR to be approved since it will be useful for NanoVDB. 

Ken mentions something we are forgetting: OpenVDB's dependency on half is hidden now: there is no in-memory representation that has a half. Any grid that is stored as a half on disk is converted to full 32 bit. This has come up from Autodesk (stuck in OpenVDB version 5 because they introduced a hack to load half from disk, keep it as half, and ray-trace it as a half in memory.) Ken mentions that, today, if you have a half grid, when you read it, it will be converted to full float. The HALF flag is only used in IO right now. Ken hasn't seen the changes that are implemented by Autodesk. Jeff sees this as another point to add a half type as a native type. 

Ken says that what we need is a way to go from 32 bit float to 16 bit float and back again. OpenVDB will work if we have that today. But Ken would like to have half as a first class citizen, so you can have a half grid in memory so that you can do ray tracing, etc. 

Dan believes that PointDataGrid supports half types: the truncate codec stores it as half in memory and when you read it, it is turned into a float. 

Jeff also agrees that we need to have half as a first class citizen (as openvdb::math::half). He proposes to have Vec3H for Vector 3 of halfs. 

Cary from ILM was asking if there is anything that they can do to make OpenEXR/Imath's half to be easier to use by OpenVDB. According to Nick: if they make it to be header only. Supposedly, if you have half float intrinsic provided by the compiler, then there is no need for a lookup table. At the end of this discussion, there is nothing that the OpenEXR/Imath group can do to make their implementation to be easier to be adopted by OpenVDB. 

Nick mentions that this PR will lock OpenVDB down to this implementation of half. 
Ken asks why we can't explore a way to have the build process pick the embedded OpenVDB's half with the other half that comes from, e.g., OpenEXR/Imath.
Nick mentions several possibilities to do this:
- generate the header at build time. 
- we can use compile defines to do this, but this is not particularly nice because downstream projects will need to use this as well. 
- we can do something weird to mimic the folder structure of OpenEXR inside of OpenVDB so we can change the include path. 

Jeff mentions that he is not sure how to inject the type-traits in half.

Ken mentions that the first step is to accept this PR so that we have a default implementation that ships with OpenVDB. In the future, we can update the half implementation with a newer version of half that comes with OpenEXR/Imath or with a version from CUDA. 

Jeff gives an argument on why this PR can be 'harmful' because it complicates the build system even more. But he believes that it is not that harmful. 

Ken asks Jeff about Houdini, since Houdini has another implementation of half? Jeff says that Houdini uses fpreal16, which will be separate from the one that is name-spaced in OpenVDB. It is not a problem right now. It will start to be a problem when we have a half grid and a developer wants to use the half implementation that is not shipped with OpenVDB. 

Jeff mentions that this PR will get rid of OpenEXR dependency. We can talk about moving it to a third_party directory. But this does not solve the problem of being able to easily link against the OpenEXR half. Is there a way to inject OpenEXR half to the openvdb::math namespace so that you can use an #ifdef in half.h/Types.h. Should we add a compile flag? Dan thinks that it is worth a try. 

Ken mentions that the current PR does not block us from including the dependency on OpenEXR/Imath in the future. Nick agrees. 

Nick thinks that we want to make the dependency to be optional, instead of removing it completely. 

Jeff mentions the problem with backward compatibility because right now OpenVDB links against OpenEXR and Types.h includes OpenEXR/half.h. The problem with this PR is that it will break someone's code if the developer depends on OpenVDB to link against OpenEXR. However, this is bad development practice, which should be avoided by including OpenEXR/half and linking against OpenEXR directly in the first place. Now, a developer may need to change the code by replacing half with math::half. Dan mentions a similar problem that comes with gcc upgrade. 

Dan summarizes what we are trying to achieve: if we were to put this embedded half into a third_party directory and always build it with the OpenVDB core library and include a mechanism that optionally switches the definition that is used by the name-spaced version, whether it is the embedded half or an external half. 

According to Nick, the issue is in installation. Nick strongly believes that we should have that option of switching the definition of half. The build-time dependency is fine. If  you want to use the embedded version of half, you need to install it, and the include path should be valid because Types.h should be able to pull in the embedded version of half. We may be able to replicate the directory structure of OpenEXR to make sure that the include path of the embedded half matches with the one that is shipped with OpenEXR, but this is not the ideal solution. Nick will continue to think about this and will come up with something that works. 

Ken asks if the current PR blocks the other approach? Nick says that it temporarily removes the dependency. Jeff says that it is a red-line we are crossing. Dan thinks that this is a good thing to figure out because it will help us to solve other dependency issues, such as blosc. 

Nick will look at this problem again and will get back. 

3) VDB 8 bugfix progress #936 (Fix CSG Intersection)
Dan is writing a more meticulous unit test around all the functionality. The existing Composite unit test does not cover everything. Merge needs to come up-stream of Composite. Dan believes that Jeff's comment on the internal node is not an issue, but he will double check this and will make sure that it is covered by the unit test. The current problem is in intersection when a child of the root node exists in one, but not the other. 

OpenVDB is still using linear search. Ken says that linear search with 8 entries is faster than binary search. Ken will take a look at how to do a search more efficiently. 

Ken will also take a look at TBB issue again. 

4) Blender update (JT)
JT has a 15 seconds update. His team has been working on making mesh to volume conversion to be better. Some of the conversion is half-baked. His group has been working on USD, Jupyter Notebook, and using OpenVDB and OpenVDBAX kernel.  

5) Min boost requirement  #935

6) AX SOP progress #931

7) Comment on USD

8) Next Meeting
Next meeting is Feb 2nd, 2021. 12pm-1pm EST (GMT-5).


