Skip to content

Synthetic benchmark for Aruco/Charuco/chessboard detection#6

Merged
asmorkalov merged 18 commits into
opencv:developfrom
AleksandrPanov:synthetic_bencmark
Sep 22, 2023
Merged

Synthetic benchmark for Aruco/Charuco/chessboard detection#6
asmorkalov merged 18 commits into
opencv:developfrom
AleksandrPanov:synthetic_bencmark

Conversation

@AleksandrPanov
Copy link
Copy Markdown
Contributor

No description provided.

@AleksandrPanov AleksandrPanov changed the base branch from master to develop September 8, 2023 01:13
@asmorkalov asmorkalov changed the title Synthetic bencmark Synthetic benchmark for Charuco board detector Sep 8, 2023
@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented Sep 8, 2023

General notes:

  • Naming is poor. Should be charcoal-thyntetic-benchmark or something meaningful.
  • Command line parameters are not used.
  • Distortion model as soon as board params should be configured with file, but bot hardcoded.

Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
return rvec, tvec


def get_coord(num_rows, num_cols, start_x=0, start_y=0):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what coordinates? please rename.

Comment on lines +161 to +168
self.cameraMatrix = np.eye(3, 3, dtype=np.float64)
self.cameraMatrix[0, 0] = img_size[0]
self.cameraMatrix[1, 1] = img_size[0]
self.cameraMatrix[0, 2] = img_size[0] / 2
self.cameraMatrix[1, 2] = img_size[0] / 2
self.distCoeffs = np.zeros((4, 1), np.float64)
self.distCoeffs[0] = -0.5012997
self.distCoeffs[1] = -0.50116057
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to run the camera parameters from file, e.g. xml generated by calibration.

Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
Comment on lines +482 to +484
print(folder)
print("detected", res[0]/res[1], "total", res[1], "distance", res[2]/max(res[1], 1),
"detected", res[3]/res[4], "total", res[4], "distance", res[5]/max(res[4], 1),)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's format it as human-readable table.

@asmorkalov
Copy link
Copy Markdown
Contributor

General notes:

  • Command line options are not obvious. I propose to use generate->detect->assess as default strategy.
  • I propose to output the final statistics as human readable table + csv for Excel. You can use pandas for it.
  • --metric does not make sense. I propose output all options in the table / csv.
  • Let's add backgroud with random noise and solt / pepper noise or Gaussian noise on top of the image. It makes the images more realistic.

@asmorkalov
Copy link
Copy Markdown
Contributor

alexander@paradox:~/tmp/tmp$ python3.8 /mnt/projects/Projects/OpenCV/opencv_benchmarks/python_benchmarks/synthetic_benchmark/synthetic_benchmark.py -o $PWD
Traceback (most recent call last):
  File "/mnt/projects/Projects/OpenCV/opencv_benchmarks/python_benchmarks/synthetic_benchmark/synthetic_benchmark.py", line 616, in <module>
    main()
  File "/mnt/projects/Projects/OpenCV/opencv_benchmarks/python_benchmarks/synthetic_benchmark/synthetic_benchmark.py", line 603, in main
    list_folders = next(os.walk(dataset_path))[1]
StopIteration

@AleksandrPanov AleksandrPanov changed the title Synthetic benchmark for Charuco board detector Synthetic objdetect benchmark for Aruco/Charuco/chessboard detect Sep 21, 2023
@AleksandrPanov AleksandrPanov changed the title Synthetic objdetect benchmark for Aruco/Charuco/chessboard detect Synthetic objdetect benchmark for Aruco/Charuco/chessboard detection Sep 21, 2023
@AleksandrPanov AleksandrPanov changed the title Synthetic objdetect benchmark for Aruco/Charuco/chessboard detection Synthetic benchmark for Aruco/Charuco/chessboard detection Sep 21, 2023
Copy link
Copy Markdown
Contributor

@asmorkalov asmorkalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread python_benchmarks/synthetic_benchmark/synthetic_benchmark.py Outdated
@asmorkalov
Copy link
Copy Markdown
Contributor

Remaining items:

  • Custom image as background.
  • CSV output and human-readable results in console.

@asmorkalov asmorkalov merged commit 51e2c5c into opencv:develop Sep 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants