findChessboardCorners performance degradation fix for some cases by thewoz · Pull Request #24558 · opencv/opencv

thewoz · 2023-11-19T14:17:28Z

In the issue #23558 an empty image was passed to the findChessboardCorners() function with the CALIB_CB_FAST_CHECK flag. The fast check failed and findChessboardCorners() wasted 15minutes finding a checkerboard that was not there in the image.

The reason is that the checkChessboardBinary() function returned true because the checkQuads() function was convinced that among all the contours found by fillQuads() were those of the checkerboard.

The problem was icvGetQuadrangleHypotheses() that incorrectly filtering the contours found.  It was passing too many contours to checkQuads(), and by the law of large numbers, checkQuads() was able to find contours that "looked" like a checkerboard between them.

What I did is to shrink the parameters in the icvGetQuadrangleHypotheses() function.  I changed the range of the ratio between the sides of the contour found to 0.5 and 1.5. More reasonable values than the previous 0.3 and 3.0.  In addition, I raised the minimum side length of the hypothetical square from 10 to 25 pixels.

After the change I tried the function on other checkerboard images and had no problems.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

asmorkalov · 2023-11-20T09:13:38Z

@thewoz Thanks a lot for the contribution. The PR introduces existing test regression:

[ RUN      ] Calib3d_ChessboardDetector.timing
C:\build\precommit_windows64\4.x\opencv\modules\ts\src\ts.cpp(618): error: Failed

	failure reason: Invalid function output
	test case #-1
	seed: 00000000000c5a7c
-----------------------------------
	LOG:
chess3.png: chessboard 1:
Error: chessboard was not detected in the image chess3.png

-----------------------------------

[  FAILED  ] Calib3d_ChessboardDetector.timing (7 ms)

thewoz · 2023-11-20T15:17:50Z

sorry where I can found the chess3.png images?

s-trinh · 2023-11-20T19:41:30Z

@thewoz
It should be this image: https://github.com/opencv/opencv_extra/blob/38d14db05e43edd9f44e673cfd296ff67a210222/testdata/cv/cameracalibration/chess3.png

thewoz · 2023-11-20T22:06:52Z

Hi,
I have test the pull request against the images:
BoardStereoL1-6.png BoardStereoR1-6.png and chess1-6.png

in

https://github.com/opencv/opencv_extra/tree/4.x/testdata/cv/cameracalibration

and the chessboard was found in each of them.
so is not clear to me the failed in the test

vpisarev · 2023-11-21T06:47:29Z

@thewoz, basically, you should clone opencv_extra repository and then set OPENCV_TEST_DATA_PATH environment variable pointing to <opencv_extra_dir>/testdata. After that just run opencv_test_calib3d and opencv_perf_calib3d.

thewoz · 2023-11-21T16:05:45Z

Hi,
I should have fixed the problem on the image chess3.png

AleksandrPanov · 2023-11-22T08:55:48Z

[ RUN      ] Calib3d_ChessboardDetector.timing
/home/ci/opencv/modules/ts/src/ts.cpp:618: Failure
Failed

	failure reason: Invalid function output
	test case #4
	seed: 00000000000c5a60
-----------------------------------
	LOG:
chess3.png: chessboard 1:
    cvCheckChessboard time s: 0.000000, us per pixel: 0.000000
    cvFindChessboard time s: 0.000000, us per pixel: 0.000000
chess1.png: chessboard 1:
    cvCheckChessboard time s: 0.000000, us per pixel: 0.000000
    cvFindChessboard time s: 0.000000, us per pixel: 0.000000
../shared/baboon.png: chessboard 0:
    cvCheckChessboard time s: 0.000000, us per pixel: 0.000000
    cvFindChessboard time s: 0.000000, us per pixel: 0.000000
../shared/box_in_scene.png: chessboard 0:
    cvCheckChessboard time s: 0.000000, us per pixel: 0.000000
    cvFindChessboard time s: 0.000000, us per pixel: 0.000000
../shared/airplane.png: chessboard 0:
    cvCheckChessboard time s: 0.000000, us per pixel: 0.000000
    cvFindChessboard time s: 0.000000, us per pixel: 0.000000
chessboard-artificial2.png: chessboard 1:
Error: chessboard was not detected in the image chessboard-artificial2.png

-----------------------------------
	CONSOLE: ...
-----------------------------------

[  FAILED  ] Calib3d_ChessboardDetector.timing (164 ms)

thewoz · 2023-11-23T15:42:29Z

Hello everyone,
everyone seems to be working now.
All tests are passed

thewoz · 2023-11-24T13:14:48Z

sorry I forgot to erase a include file that I use for the debug

thewoz · 2023-12-01T10:46:17Z

Hello everyone is there anything else I can do ?

asmorkalov · 2023-12-01T11:46:34Z

@vpisarev could you take a look too?

thewoz · 2023-12-14T08:37:18Z

is there anything i can do?

vpisarev · 2023-12-19T07:54:39Z

@thewoz, thank you! I think, the patch can be merged.

AleksandrPanov

need to fix extra \n
what is the performance? reabase to #24605 (comment), this PR improves performance.
I suggest to use approxPolyDP() instead box_area_covered (check line 1795 in calibinit.cpp)

AleksandrPanov · 2023-11-24T13:17:32Z

+        float box_area = box.size.width * box.size.height;
+        if(contourArea(c) < (box_area_covered * box_area))
+        {
+          continue;
+        }
+


I think it's a good idea. We need to understand how much the found contour looks like a square of a chessboard. We could use approxPolyDP() for this. We could use algorithm from _findMarkerContours() in ArucoDetector:

// check is square and is convex vector<Point> approxCurve; approxPolyDP(contours[i], approxCurve, double(contours[i].size()) * accuracyRate, true); if(approxCurve.size() != 4 || !isContourConvex(approxCurve)) continue; // check min distance between corners double minDistSq = max(contoursImg.cols, contoursImg.rows) * max(contoursImg.cols, contoursImg.rows); for(int j = 0; j < 4; j++) { double d = (double)(approxCurve[j].x - approxCurve[(j + 1) % 4].x) * (double)(approxCurve[j].x - approxCurve[(j + 1) % 4].x) + (double)(approxCurve[j].y - approxCurve[(j + 1) % 4].y) * (double)(approxCurve[j].y - approxCurve[(j + 1) % 4].y); minDistSq = min(minDistSq, d); } double minCornerDistancePixels = double(contours[i].size()) * minCornerDistanceRate; if(minDistSq < minCornerDistancePixels * minCornerDistancePixels) continue;

Hi @AleksandrPanov I never perform a rebase.
Can you help me? Sorry about that...
for now I fixed the returns and tested with approxPolyDP() and everything seems to be working.

checkout your branch to 4.x, update your 4.x branch from https://github.com/opencv/opencv, checkout your branch to fix-issue-23558, rebase and resolve conflicts:

git checkout 4.x git pull https://github.com/opencv/opencv git checkout fix-issue-23558 git rebase 4.x git push -f

fix tab arrow issue

Add weights yolov3 in models.yml opencv#24496 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [X] I agree to contribute to the project under Apache 2 License. - [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [X] The PR is proposed to the proper branch - [X] There is a reference to the original bug report and related work - [X] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [X] The feature is well documented and sample code can be built with the project CMake I don't know if this action is necessary, or the previous PR scale for the brach master. Thanks.

Handle huge images in IPP distanceTransform opencv#24535 ### Pull Request Readiness Checklist * Do not use IPP for huge Mat (reproduced with opencv#23895 (comment) on `DIST_MASK_5`) I have observed two types of errors on the reproducer from the issue: 1. When `temp` is not allocated: ``` Thread 1 "app" received signal SIGSEGV, Segmentation fault. 0x00007ffff65dc755 in icv_l9_ownDistanceTransform_5x5_8u32f_C1R_21B_g9e9 () from /home/dkurtaev/opencv_install/bin/../lib/libopencv_imgproc.so.408 (gdb) bt #0 0x00007ffff65dc755 in icv_l9_ownDistanceTransform_5x5_8u32f_C1R_21B_g9e9 () from /home/dkurtaev/opencv_install/bin/../lib/libopencv_imgproc.so.408 opencv#1 0x00007ffff659e8df in icv_l9_ippiDistanceTransform_5x5_8u32f_C1R () from /home/dkurtaev/opencv_install/bin/../lib/libopencv_imgproc.so.408 opencv#2 0x00007ffff5c390f0 in cv::distanceTransform (_src=..., _dst=..., _labels=..., distType=2, maskSize=5, labelType=1) at /home/dkurtaev/opencv/modules/imgproc/src/distransform.cpp:854 opencv#3 0x00007ffff5c396ef in cv::distanceTransform (_src=..., _dst=..., distanceType=2, maskSize=5, dstType=5) at /home/dkurtaev/opencv/modules/imgproc/src/distransform.cpp:903 opencv#4 0x000055555555669e in main (argc=1, argv=0x7fffffffdef8) at /home/dkurtaev/main.cpp:18 ``` 2. When we keep `temp` allocated every time: ``` OpenCV(4.8.0-dev) Error: Assertion failed (udata < (uchar*)ptr && ((uchar*)ptr - udata) <= (ptrdiff_t)(sizeof(void*)+64)) in fastFree, file /home/dkurtaev/opencv/modules/core/src/alloc.cpp, line 191 terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.8.0-dev) /home/dkurtaev/opencv/modules/core/src/alloc.cpp:191: error: (-215:Assertion failed) udata < (uchar*)ptr && ((uchar*)ptr - udata) <= (ptrdiff_t)(sizeof(void*)+64) in function 'fastFree' ``` * Try enable IPP for 3x3 (see opencv#15904) * Reduce memory footprint with IPP See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

…n and bug fix (opencv#24462) * add hardswish for cann * gemm cann bug fix * fix indentation * cann: add layer norm * cann: add instance norm * add supportBackend * cann: layer norm does not support axis=-1 due to 1d mat issue * disable instance norm for now * fix doc * remove tensor desc initialization for 1D tensor

…e-selection-onnxrt-directml G-API: Advanced device selection for ONNX DirectML Execution Provider opencv#24060 ### Overview Extend `cv::gapi::onnx::ep::DirectML` to accept `adapter name` as `ctor` parameter in order to select execution device by `name`. E.g: ``` pp.cfgAddExecutionProvider(cv::gapi::onnx::ep::DirectML("Intel Graphics")); ``` ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [ ] I agree to contribute to the project under Apache 2 License. - [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [ ] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

Using cv2 dnn interface to run yolov8 model opencv#24396 This is a sample code for using opencv dnn interface to run ultralytics yolov8 model for object detection. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [X] I agree to contribute to the project under Apache 2 License. - [X] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [X] The PR is proposed to the proper branch - [] There is a reference to the original bug report and related work - [] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [] The feature is well documented and sample code can be built with the project CMake

…_gemm Fast gemm for einsum opencv#24509 ## This PR adds performance tests for Einsum Layer with FastGemm. See below results of performance test on different inputs ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

…on-estimator Bugfix/qrcode version estimator opencv#24364 Fixes opencv#24366 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake

Updated Android samples for modern Android studio. Added OpenCV from Maven support. opencv#24473 Updated samples for recent Android studio: - added namespace field that is required in build.gradle files - replaced _switch_ by _if-else_ because it doesn't work with constants from resources - added missed log library dependency in face-detection/jni/CMakeLists.txt - use local.properties to define NDK location Added support for OpenCV from Maven. Now you can choose 3 possible sources of OpenCV lib in settings.gradle: SDK path, local Maven repository, public Maven repository. (Creating Maven repository from SDK is added here opencv#24456 ) There are differences in project configs for SDK and Maven versions: - different dependencies in build.gradle - different OpenCV library names in CMakeLists.txt - SDK version requires OpenCV_DIR definition Requires: - opencv/ci-gha-workflow#124 - opencv-infrastructure/opencv-gha-dockerfile#26

thewoz · 2024-01-09T09:49:31Z

Hello everyone,
I was trying the modification suggested by @AleksandrPanov to try to use the approxPolyDP() function and it wasn't working too well.
Anyway doing some testing I realized that today's version of 4.x no longer gives the issue #23558
I think that the pull #24605, has solved this #23558
At this point I would say that this pull of mine is useless.

AleksandrPanov · 2024-01-10T12:11:25Z

@thewoz, I tried to use the approxPolyDP() func too, and it really wasn't working too well. You could try to call this function twice, but it doesn't look like a good solution.

Now I think that use minAreaRect and box_area_covered is better. Thanks for the good idea.

thewoz · 2024-01-10T22:16:35Z

Hi @AleksandrPanov thank you for your time..
and yes, we could try to run approxPolyDP() twice (I haven't tried yet).
However, my concern is that approxPolyDP() is a bit too sensitive to the sharpeness of the edges of the checkerboard squares.
My idea is that using minAreaRect and box_area_covered would make the process a little less sensitive to this problematic.
Anyway, the reason I tried to implement this solution is to solve problem #23558, in which too many contours were found in an image and checkChessboardBinary() thought there was a checkerboard.
So I wanted to try to implement a method that would remove those contours that could not be considered square.
Up to now the pull request #24605 seems to have solved the problem #23558

AleksandrPanov · 2024-01-15T10:23:01Z

@thewoz, there are not enough tests in OpenCV for findchessboardcorners() and therefore algorithm changes can cause regressions that we may not catch.
We merged PR #24605 because it just adds optimizations without changing the algorithm.

We want to expand the test suite, but this is a big task. After that, we will try to improve the code based on your ideas.

thewoz · 2024-01-15T12:17:50Z

Hi @AleksandrPanov,
I completely understand what you mean and I fully agree with you.
Thank you for all the work

thewoz changed the title ~~Update checkchessboard.cpp~~ fix Issue 23558 Nov 19, 2023

asmorkalov changed the title ~~fix Issue 23558~~ findChessboardCorners performance degradation fix for some cases Nov 20, 2023

asmorkalov added bug optimization category: calib3d labels Nov 20, 2023

asmorkalov requested a review from AleksandrPanov November 20, 2023 07:43

asmorkalov added this to the 4.9.0 milestone Nov 20, 2023

asmorkalov requested a review from vpisarev December 1, 2023 11:46

vpisarev approved these changes Dec 19, 2023

View reviewed changes

AleksandrPanov requested changes Dec 19, 2023

View reviewed changes

asmorkalov modified the milestones: 4.9.0, 4.10.0 Dec 22, 2023

YusukeKameda and others added 5 commits January 4, 2024 11:56

doxygen: Fix ImwriteFlags documentation misalignment

d04e974

videoio: fix v4l2 test on older platforms (centos)

a278c9e

fix: recursively re-export nested submodules

2a0ac79

Add Ubuntu 22.04 to CI.

e898e94

Fixed tests execution with Python 2.7

3a5993c

fengyuentau and others added 20 commits January 4, 2024 11:58

remove filter

b83ad02

Update window_QT.cpp

f8d81ec

fix tab arrow issue

remove filter fix

5f9435d

Update checkchessboard.cpp

2d5164e

Update checkchessboard.cpp

2b3b779

Update checkchessboard.cpp

ef375dd

Update checkchessboard.cpp

f72cfab

removed trailing whitespace

b95805b

remove trailing whitespace

4e48714

fix extra \n

d6275ed

use approxPolyDP

eeec18e

Update checkchessboard.cpp

45e047f

thewoz force-pushed the fix-issue-23558 branch from 28fbca0 to 45e047f Compare January 4, 2024 10:58

thewoz added 3 commits January 5, 2024 10:12

Update lpsolver.cpp

9eb307b

fix issue

55ccf82

Merge remote-tracking branch 'upstream/4.x' into fix-issue-23558

469d62f

thewoz closed this Jan 9, 2024

Uh oh!

Conversation

thewoz commented Nov 19, 2023 • edited by asmorkalov Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

asmorkalov commented Nov 20, 2023

Uh oh!

thewoz commented Nov 20, 2023

Uh oh!

s-trinh commented Nov 20, 2023

Uh oh!

thewoz commented Nov 20, 2023

Uh oh!

vpisarev commented Nov 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thewoz commented Nov 21, 2023

Uh oh!

AleksandrPanov commented Nov 22, 2023

Uh oh!

thewoz commented Nov 23, 2023

Uh oh!

thewoz commented Nov 24, 2023

Uh oh!

thewoz commented Dec 1, 2023

Uh oh!

asmorkalov commented Dec 1, 2023

Uh oh!

thewoz commented Dec 14, 2023

Uh oh!

vpisarev commented Dec 19, 2023

Uh oh!

AleksandrPanov left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AleksandrPanov Nov 24, 2023

Choose a reason for hiding this comment

Uh oh!

thewoz Dec 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AleksandrPanov Dec 22, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thewoz commented Jan 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AleksandrPanov commented Jan 10, 2024

Uh oh!

thewoz commented Jan 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AleksandrPanov commented Jan 15, 2024

Uh oh!

thewoz commented Jan 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

thewoz commented Nov 19, 2023 •

edited by asmorkalov

Loading

vpisarev commented Nov 21, 2023 •

edited

Loading

AleksandrPanov left a comment •

edited

Loading

thewoz Dec 20, 2023 •

edited

Loading

thewoz commented Jan 9, 2024 •

edited

Loading

thewoz commented Jan 10, 2024 •

edited

Loading