[GSoC] Update octree methods and create frames for PCC by starga2er777 · Pull Request #23985 · opencv/opencv

starga2er777 · 2023-07-13T08:24:25Z

PR for GSoC Point Cloud Compression

Issue for GSoC 2023

We are updating the Octree method create() by using OctreeKey: Through voxelization, directly calculate the leaf nodes that the point cloud belongs to, and omit the judgment whether the point cloud is in the range when inserted. The index of the child node is calculated by bit operation.
We are also introducing a new header file pcc.h (Point Cloud Compression) with API framework.
We added tests for restoring point clouds from an octree.
Currently, the features related to octree creation and point cloud compression are part of the internal API, which means they are not directly accessible to users. However, our plan for the future is to include only the 'PointCloudCompression' class in the 'opencv2/3d.hpp' header file. This will provide an interface for utilizing the point cloud compression functionality.

The previous PR of this was closed due to repo name conflicts, therefore we resubmitted in this PR.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

TsingYiPainter · 2023-07-13T08:51:41Z

@opencv-alalek @asmorkalov, Hi, although I changed the name of the repo (Its name is now opencv, the original name was opencv_contrib) and make a new PR, the build still failed.

As you can see the repo name is still opencv_contrib. How can we do? Should we delete this repo, fork again and submit PR?

opencv-alalek · 2023-07-13T15:43:37Z

Because GitHub keep alias for old repo name.

Try to create empty opencv_contrib repo to reset this alias (and may be delete it after that).

vpisarev · 2023-07-26T06:33:53Z

    * @param maxDepth The max depth of the Octree. The maxDepth > -1.
    */
-    explicit Octree(int maxDepth);
+    explicit Octree(size_t maxDepth);


why change API w/o any good reason?

Because I think maxDepth should be non-negative, and part of the subsequent code compares maxDepth to unsigned integers, I adopted this approach to avoid potential problems.

In addition, I forgot to change the corresponding explanation, which is my negligence.

It's better to have int here since, among other reasons, it has better compatibility with our wrapper generators for Python and Java.

vpisarev · 2023-07-26T06:35:04Z

+    * @param resolution The size of the octree leaf node.
    */
-    Octree(const std::vector<Point3f> &pointCloud, int maxDepth);
+    Octree(const std::vector<Point3f> &pointCloud, double resolution);


maybe both resolution and maxDepth could be specified or one of them; why remove maxDepth?

Perhaps it's reasonable to retain both options.

The resolution denotes the size of the leaf node. I opted for resolution as it aligns well with point cloud compression, where preserving information at this precision is sufficient.

For the current octree functions, such as KNN search, it might be better to retain both methods. Simultaneously, the two can be calculated from one to the other.

What's your opinion?

Since octree is used not only for point cloud compression, I'd propose you to keep both constructors (resolution and maxDepth)

vpisarev · 2023-07-26T06:36:24Z

@savuor, could you please review it?

asmorkalov · 2023-10-05T07:02:57Z

@savuor Friendly reminder.

savuor

Please take a look at my comments, looks like there are several things to fix

savuor · 2024-01-26T03:13:51Z

+    * @param resolution The size of the octree leaf node.
    */
-    Octree(const std::vector<Point3f> &pointCloud, int maxDepth);
+    Octree(const std::vector<Point3f> &pointCloud, double resolution);


Since octree is used not only for point cloud compression, I'd propose you to keep both constructors (resolution and maxDepth)

savuor · 2024-01-26T03:22:43Z

+    * @param restorePointCloud The output point cloud data.
+    * @param restoreColor The color attribute of point cloud data
+    */
+    void getPointCloudByOctree(std::vector<Point3f> &restorePointCloud, std::vector<Point3f> &restoreColor);


Should this method calculate an average color for the point in the same leaf?
If not, it should be said explicitly in the docs to the function

savuor · 2024-01-26T03:33:21Z

 }

-bool insertPointRecurse( Ptr<OctreeNode>& _node,  const Point3f& point, int maxDepth)
+bool insertPointRecurse( Ptr<OctreeNode>& _node,  const Point3f& point,const Point3f &color, size_t maxDepth,const OctreeKey &key,


Looks like this function always returns true, don't need to return anything at all

savuor · 2024-01-26T03:44:27Z

    EXPECT_NO_THROW(treeTest.radiusNNSearch(restPoint, radius, outputPoints, outputSquareDist));
+    EXPECT_EQ(outputPoints.size(),(unsigned int)5);

+    //The outputPoints should be unordered, so in some resolution, this test will fail because it specified the order of output.


If this is so, maybe the points should be sorted by distance first?

savuor · 2024-01-26T03:47:18Z

Looks like this file belongs to next PR about point cloud compression (probably #24197), should it be here?

savuor · 2024-01-26T08:32:05Z

A decision was to fix this PR by me

broweigit · 2024-02-02T05:58:29Z

Sorry for the late reply. I'm still here working on this PointCloudCompression commitment since GSOC. We've made a variety of experiment on PCC algorithms but the most significant improvement lies on a structural improvement on the OctreeNode class and the usage of Zlib.
We now hold a functional PCC code developed in OpenCV(and usable, better than PCL, I guess?). Yet I just want you to know that we try to make separate PRs to push limited workload stage by stage.
Looks like some fix have been made. Since you know the exact demand that opencv needs. maybe we shall wait until this PR is stabled, then continue our next part? But if there's anything we can do, please contact us! Thanks.

broweigit · 2024-02-02T06:10:29Z

Our further PCC code should based on this PR, I shall make some adjustment on latter PCC framwork after you finished. Is it?

savuor · 2024-02-19T19:10:24Z

@broweigit Hi there! I'm trying to fix this PR according to the notes I've made (and according to my personal understand of how this should be done :) ). I will invite you to review the changes I've made, along with other contributors.
Right after this PR is done, I'm going to work on Point Cloud Compression PR (#24197) in the same manner.

savuor · 2024-02-22T23:42:12Z

@broweigit I've changed API and some implementation details, can you take a look?

asmorkalov · 2024-02-25T07:50:26Z

+     * @param colors color attribute of point cloud in the same 3-channel float format
+     * @return resulting Octree
+     */
+    static Octree createWithDepth(int maxDepth, InputArray pointCloud, InputArray colors = noArray());


Input arrays usually go first in the public API.

This was made intentionally to reduce amount of overloaded methods

asmorkalov · 2024-02-25T07:58:16Z

+    for (int i = 0; i < (int)goldVals.size(); i++, it++)
+    {
+        Point3f p = it->second;
+        EXPECT_FLOAT_EQ(goldVals[i].x, p.x);
+        EXPECT_FLOAT_EQ(goldVals[i].y, p.y);
+    }


It could be just NORM_INF

I propose to leave it as it is, using NORM_INF looks cumbersome here

broweigit · 2024-02-26T08:18:52Z

@broweigit I've changed API and some implementation details, can you take a look?

Yep, checked. no problem with the changes for far. Thanks

asmorkalov

👍

asmorkalov · 2024-02-28T07:05:17Z

@vpisarev @TsingYiPainter Do you have any remarks?

starga2er777 mentioned this pull request Jul 13, 2023

[GSoC] Update octree methods and create frame for PCC #23959

Closed

6 tasks

WanliZhong added GSoC category: 3d module labels Jul 13, 2023

WanliZhong requested a review from savuor July 13, 2023 08:27

vpisarev reviewed Jul 26, 2023

View reviewed changes

WanliZhong mentioned this pull request Aug 27, 2023

Pointcloud Compression Implementation in 3d module. #24197

Open

6 tasks

asmorkalov added this to the 5.0 milestone Oct 5, 2023

savuor requested changes Jan 26, 2024

View reviewed changes

TsingYiPainter and others added 13 commits February 19, 2024 20:10

Refactor relevant code in 3d module

951adfa

Resolve warning

2b2959e

Create pcc.h, add basic API framework.

5beb84e

Resolve warning

f18c4c5

Resolve warning

216d4b9

Resolve warning

133cd8e

Resolve warning

ba30b4d

Resolve warning

17265c6

tests fixed

6677be7

refactoring pt.1

d20cc7a

isPointInBound

1d71f4f

overlap

b50ec1f

insert/delete

2d61c5c

Rostislav Vasilikhin added 8 commits February 19, 2024 20:10

radiusNNsearch

c82f9ca

knnSearch

408b1df

maxDepth: size_t -> int

2a26655

updating API

60a5699

constructors & destructors

21d3867

API reworked, in progress

9f27cf9

colors, args and other fixes

f83c7d5

knn fix

2a5030d

savuor force-pushed the pcc branch from 54d4c59 to 2a5030d Compare February 22, 2024 23:41

Rostislav Vasilikhin added 2 commits February 23, 2024 14:43

docs and warnings fixed

ede75cf

try to fix win warning

0418c4b

asmorkalov reviewed Feb 25, 2024

View reviewed changes

asmorkalov reviewed Feb 26, 2024

View reviewed changes

Comment thread modules/3d/src/octree.cpp Outdated

Rostislav Vasilikhin and others added 9 commits February 27, 2024 21:56

bindings fix

688d6d3

signature fixes

712f756

sorting values

0f015e9

getPoint w/o recursion

c29ce2d

index() w/o recursion

90ea54b

insert w/o recursion

f1f17d7

radius NN search w/o recursion

346bd67

minors

4b37aac

Review fixes.

a1cc822

asmorkalov approved these changes Feb 28, 2024

View reviewed changes

asmorkalov merged commit 0e47b05 into opencv:5.x Feb 29, 2024

Uh oh!

Conversation

starga2er777 commented Jul 13, 2023

PR for GSoC Point Cloud Compression

Pull Request Readiness Checklist

Uh oh!

TsingYiPainter commented Jul 13, 2023

Uh oh!

opencv-alalek commented Jul 13, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vpisarev commented Jul 26, 2023

Uh oh!

asmorkalov commented Oct 5, 2023

Uh oh!

savuor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

savuor commented Jan 26, 2024

Uh oh!

broweigit commented Feb 2, 2024

Uh oh!

broweigit commented Feb 2, 2024

Uh oh!

savuor commented Feb 19, 2024

Uh oh!

savuor commented Feb 22, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

broweigit commented Feb 26, 2024

Uh oh!

Uh oh!

asmorkalov left a comment

Choose a reason for hiding this comment

Uh oh!

asmorkalov commented Feb 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants