{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
" \n",
" \n",
" ![](\"https://user-images.githubusercontent.com/25985824/104791629-6e618700-5769-11eb-857f-d176b37d2496.png\") \n",
" Try in Google Colab\n",
" \n",
" | \n",
" \n",
" \n",
" ![](\"https://user-images.githubusercontent.com/25985824/104791634-6efa1d80-5769-11eb-8a4c-71d6cb53ccf0.png\") \n",
" Share via nbviewer\n",
" \n",
" | \n",
" \n",
" \n",
" ![](\"https://user-images.githubusercontent.com/25985824/104791633-6efa1d80-5769-11eb-8ee3-4b2123fe4b66.png\") \n",
" View on GitHub\n",
" \n",
" | \n",
" \n",
" \n",
" ![](\"https://user-images.githubusercontent.com/25985824/104792428-60f9cc00-576c-11eb-95a4-5709d803023a.png\") \n",
" Download notebook\n",
" \n",
" | \n",
"
\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Working with Feature Points\n",
"\n",
"This notebook demonstrates how to compute feature points for images and visualize them in [FiftyOne](https://github.com/voxel51/fiftyone).\n",
"\n",
"Specifically, we'll compute [ORB features](https://docs.opencv.org/3.4/d1/d89/tutorial_py_orb.html) using OpenCV, which is installed by default along with FiftyOne."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"Make sure you have [FiftyOne installed](https://voxel51.com/docs/fiftyone/getting_started/install.html):\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install fiftyone"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then collect a directory of images that you'd like to process:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"IMAGES_DIR = \"/path/to/images\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create a dataset\n",
"\n",
"Loading the images into a [FiftyOne dataset](https://voxel51.com/docs/fiftyone/user_guide/basics.html) is easy:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import fiftyone as fo"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" 100% |██████████████████████████████| 13/13 [20.7ms elapsed, 0s remaining, 629.0 samples/s] \n"
]
}
],
"source": [
"dataset = fo.Dataset.from_images_dir(IMAGES_DIR)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Name: 2021.01.21.00.16.24\n",
"Media type: image\n",
"Num samples: 13\n",
"Persistent: False\n",
"Info: {}\n",
"Tags: []\n",
"Sample fields:\n",
" filepath: fiftyone.core.fields.StringField\n",
" tags: fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)\n",
" metadata: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)\n"
]
}
],
"source": [
"print(dataset)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Compute feature points\n",
"\n",
"Now let's add the feature points to the dataset.\n",
"\n",
"We'll start by instantiating an OpenCV ORB detector:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"\n",
"detector = cv2.ORB_create(nfeatures=500)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and then define a function that will extract the keypoints for an image:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"def get_keypoints(image_path, detector):\n",
" img = cv2.imread(image_path)\n",
" height = img.shape[0]\n",
" width = img.shape[1]\n",
" keypoints = detector.detect(img, None)\n",
" return [(kp.pt[0] / width, kp.pt[1] / height) for kp in keypoints] # convert to [0, 1] x [0, 1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can write a simple loop to add the keypoints to our FiftyOne dataset in a new `keypoints` field of each sample using FiftyOne's [keypoints label type](https://voxel51.com/docs/fiftyone/user_guide/using_datasets.html#keypoints):"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"for sample in dataset:\n",
" points = get_keypoints(sample.filepath, detector)\n",
" sample[\"keypoints\"] = fo.Keypoint(points=points)\n",
" sample.save()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Notice that the dataset now has a `keypoints` field containing the ORB featuure points:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Name: 2021.01.21.00.16.24\n",
"Media type: image\n",
"Num samples: 13\n",
"Persistent: False\n",
"Info: {}\n",
"Tags: []\n",
"Sample fields:\n",
" filepath: fiftyone.core.fields.StringField\n",
" tags: fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)\n",
" metadata: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)\n",
" keypoints: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Keypoint)\n"
]
}
],
"source": [
"print(dataset)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Visualize in the App\n",
"\n",
"Now we're ready to launch the [FiftyOne App](https://voxel51.com/docs/fiftyone/user_guide/app.html) and visualize the keypoints:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
"
\n",
" \n",
"
\n",
"
\n",
"