ABSTRACT
Sudden Infant Death Syndrome, also known as SIDS, is the leading cause of mortality in infants from one month to one year of age. We propose a video – based baby monitoring system with Internet of Things (IoT) capabilities to help shorten the response time of SIDS cases. Using a video amplification technique developed at MIT dubbed “Eulerian Magnification” to amplify subtle movements we can compare pixel color differences in frames for breathing detection in a recorded video of a baby. In the event that abnormal movement is detected from the baby an alarm will be generated to notify the parents or guardians.
DESIGN AND IMPLEMENTATION
Figure 2.1: The Samsung Artik 10 board
The team was given an Artik 10 board to be used in the implementation of the project. The Artik 10 is an IoT board released by Samsung with a octa-core architecture and a quad processor. It also has a 32 bit ARM CPU that makes it ideal for high computation activities such as image processing, 3D graphics, and drones. The board has DRAM and flash memory capabilities while also supporting Wi-Fi, Bluetooth, and Zigbee. In addition the Artik has USB and MIPI camera compatibility and a Mali T628 MP6 graphic processing unit that can show HD videos at 120 frames per second. Finally, the physical board comes at a relatively small size of 29 x 39 x 1.3 mm.
Figure 2.2: Final Design Pictorial Diagram
Figure 2.2 is a pictorial diagram that represents the final operation of our system. The system can be broken down into three major subcategories: the video acquisition portion, the breath detection algorithm portion, and the alarm portion. The first system subcategory, the video acquisition portion, is represented by the first two sequential images in the diagram, which are a video camera and the Artik board. A night vision video camera is connected to the Artik board and used to capture a video of the baby. The second system subcategory, the motion detection algorithm portion, is represented by the third and fourth images in the sequence.
Figure 2.3: Night Vision Camera
Since the intent of the project was to record footage of a baby sleeping during the night a night vision camera was necessary. The team has selected to use a 5 MP CMOS UVC USB 8 Camera module with a 2.1 mm lens and 24 infrared LEDs. The camera also has a high frame rate of 30 fps and a 1 280×720 resolution in addition to coming attached to a 1m cable. Originally the team looked for MIPI cameras but chose a USB camera in the end due to a wider variety of night vision options and compatibility. The camera chosen is a relatively high quality piece of equipment as the team was initially not sure what the optimum resolution and fps would be to work at.
ROADBLOCKS
Internet Connectivity
One of the first issues that arose when working with the Artik board was connecting to the internet. Since the team initially wanted to store data in a cloud the board would need to be able to connect to the internet. After connecting the Wi – Fi antenna to the Artik board and launching PuTTY the wpa_supplicant.conffile can be modified to allow access to the internet. In PuTTY a Wi – Fi scan command can be entered to see the available networks. Initially when the scan command was run no networks appeared. The no network problem was remedied by updating the Artik Image.
- Streaming
- Running Eulerian on Artik
- Using Twilio for SMS
BREATH DETECTION ALGORITHM TESTING AND RESULTS
Figure 4.4: Chest Movement Plot from 13 Different Videos
To simulate different scenarios the team created 13 different sample videos for analysis. Some of the videos contained breathing and others were videos containing no movement at all. The thirteen different videos after Eulerian magnification is plotted in figure 4.4. Even though many of the videos displayed similar sinusoidal behavior, the magnitudes of each plot vastly differed, partially due to different lighting conditions. The vast differences in magnitudes makes it harder to generate an algorithm that will work in multiple scenarios as some of the videos with no movement, or “noise”, had larger magnitudes than videos with breathing.
Figure 4.13: Text Messages Indicating Normal and Abnormal Breathing
By utilizing the threshold level a text file is able to be generated from the video indicating the state of the baby. If the baby is breathing okay then the text file would contain a “1” and if an emergency is detected then the text file would contain a “0”. With some outside help the team was able to come up with Java code that would read the contents of the file and utilize Twilio in order to send a message to a desired phone number.
The code was run in the program Eclipse, which is the most commonly used Java Integrated Development Environment. Upon running the code a text message would be received to the desired phone number indicating if normal or abnormal breathing was detected, as shown in figure 4.13. Sending messages when normal breathing is detected would not be necessary, but was done during testing to see if there was proper functionality.
PROFESSIONAL ISSUES AND CONSTRAINTS
Ethics
Ethics is ultimately one of the main driving forces behind engineering. The goal of engineering is to use science and math principles in a way that will improve the lives of others. However, not all engineers will use their mental capabilities in a manner that is good for the race, which brings upon the importance of having ethical consideration and discussion. Stopping for a moment to consider the ethical impacts of a product may actually end up changing the way the product is designed.
- Ethical Justification
- Sustainability
- Environmental Sustainability
- Social Sustainability
- Economic Sustainability
- Health and Safety
- Manufacturability and Usability
- Civic Engagement
CONCLUSION
Unfortunately not all of the requirements initially set by the team could be met. The main requirement that could not be met was having the baby monitoring system run in real time. The system is not ready for use in a real environment since it does not run in real time. As the real time portion of the project was not completed the long duration run time of the system could not accurately be tested or measured either. A work around of capturing short clips was used in order to test the functionality of the rest of the project.
Despite the incompletion of the having an automated system run in real time, the goal of detecting breathing in a video was successfully accomplished. Being able to track and monitor a person’s breathing rate with a video camera has potentially huge applications within the medical field. In addition to monitoring the subject’s breathing rate for abnormal patterns the team was also able to generate a text message in the event of an emergency. A breath detector would not be very useful in its own right if it constantly had to be monitored, so utilizing SMS smoothens the process.
As stated above the main feature that needs to be worked on is the automation and real time performance of the system. In addition to the real time and automation another issue that needs to be dealt with is testing the system on babies. Getting access to and testing on babies is an issue that was avoided by testing the system on the team members. The principle concept remains the same, but some threshold values for breath detection will likely need to be changed due to anatomical differences in newborns.
Source: Santa Clara University
Authors: Xiaoting Liu | Kyle Takeuchi
>> 200+ IoT Led Projects for Final Year Students
>> IoT Projects in Medical Field for Final Year Students