Estimating energy consumption of applications using thermal imaging

Pervasive Data Science Seminar - Supervisor: Huber Flores

Hashim Hashimov

Nowadays, due to ergonomic optimization in designs, smart and IoT devices have batteries that cannot be detached. Thermal imaging running in smartphone cameras can be used to identify the level of heat in an area by looking at the invisible spectrum of light. In this project, we analyze whether it is possible to make a correlation between the energy consumption of applications running in devices and measurements obtained by thermal imaging when pointing the camera at the device

Repository Structure

This repository consists of 2 main folders:

• Web Project
• Jupyter Notebooks

Jupyter Notebooks folder basically contains the notebooks which had been developed during the seminar to try-fail purposes to reach to the new milestone. Web project folder is the main deliverable for the seminar which allows user to upload the thermal picture of phone and receive the heat footprint. x, type some text into the left window and watch the results in the right.

Strategies

Firstly, application resizes the picture into %50 less dimensions. Secondly, first extraction happens and it tries to extract phone part from the image with the help of following configurations

cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU + cv2.THRESH_BINARY)[1]

Next step is extracting the rectangle which covers heated part of phone:

thresh = cv2.threshold(gray, 179, 255, cv2.THRESH_BINARY)[1]
# Find contour and sort by contour area
cnts = cv2.findContours(
    thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)

The last step is to replace the non-heat parts with white color and get only the heat part.

ret, mask = cv2.threshold(gray, 140, 255, cv2.THRESH_BINARY)
kernel = np.ones((9, 9), np.uint8)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)

# put mask into alpha channel of result
result = img.copy()
result = cv2.cvtColor(result, cv2.COLOR_BGR2BGRA)
result[:, :, 3] = mask
result = cv2.bitwise_and(result, result, mask=mask)

And user will receive the image which has only heat part.

Furher developments

For thesis, this project will be extended to return approximate energy consumption. In jupyter notebooks folder, notebooks has the code to process the matrices generated by the thermal logger app.