Locating Where a Sound is Coming From

For my masters year, half the marks came from one module, the masters project. Being a team effort, we were in a group of three. Putting our heads together, and taking ideas from lecturers, we made a list of potential projects. We knew for one that I wanted to be making hardware, and the other two wanted to use/learn machine learning and maybe FPGA’s. After much deliberation we decided to make a project that listened for a sound, and using time difference of arrival worked out where the sound came from. This post is mostly about the hardware and circuitry designed for the project.

The final board for our masters project. Contains four amplifier sections for the microphones and a micro controller with USB interface.

With a world with a big focus on safety in public places, we thought it would be a good product for the security industry, potentially with links to smart cities. Imagine a shopping center, somewhere with lots of security already. They tend to have lots of security cameras, alarm systems and a dedicated guard. This isn’t uncommon in big public places/attractions, especially in the UK. Sports stadiums, train stations and museums are always looking for new ways to protect themselves and isolate problems. The example that inspired us was the horrendous shooting in Las Vegas at a concert in October 2017, just as we were picking projects. The main problem was that the security services did not know where the shooter was, meaning it took longer to get to him. If they had a system like we envisaged, the microphones would pick up the sound and triangulate it. The location could then be sent to relevant authorities to use.

The front page of the New York times just days after the Las Vegas shooting

To start with we needed microphones. We didn’t need to reinvent the wheel, and microphones can be easily bought off the shelf. For ease we used standard stage microphones, that had 3-pin XLR outputs. Although we had been warned that they would not work they had a good omnidirectional pattern, and had lots of good documentation. One issue with them is the output is balanced, which means it needs to go through a pre-amp. To get an idea of what a balanced signal is, imagine a ground connection and two signals. The two signals are the same, but one is inverted. This means when it travels down the cable it is much less susceptible to interference. This is part of the reason we liked using stage rated equipment, as sound engineers have already worked out issues with transporting sound signals long distances through noisy situations. We concluded from research that the signals could reach over 100m, which was the number we were aiming for.

One of the pre-amplifier sections used on the board, using four operational amplifiers.

Once the signal got to the box it needed to be converted to a signal that could be read by an ADC. To do this we used an INA217, a pre-amp designed for basically this purpose. An instrument amplifier, it measures the difference between the signals and amplifies them, outputting a voltage with reference to ground. The signal from the microphone is tiny, in the tens of milivolts range, so it needed some dramatic amplification to get it near the 5V ADC. The INA217 did a good job but we put a second stage amplifier to give it the extra push, as very large gains can sometimes be bad for a number of reasons. We used an OP07D but if we were to do it again we would get a rail-to-rail to get better results. This amp had a pot as one of the gain resistors so that we could easily trim the gain depending on test. Finally, the signal at this point sat between -2.5V and +2.5V so we needed to shift it up so it was between 0 and 5V. This was done with a simple shift circuit and an amplifier. We used another OP07D to make buying easier.

Me manufacturing the PCB, at this point I was inspecting to see how well it had been soldered.

From here the signal gets read by the 12 bit ADC in an STM32 microcontroller. It then streams the data via the USB to a PC where MATLAB picks it up. This is where my knowledge is a bit lacking as I did not make it. In essence MATLAB uses a machine learning algorithm that had listened to over 1000 gunshots, screams and explosions. It has categorized them, and used a number of features to notice the difference. Then when playing a new sound of one of these things (not heard by it before) it categorizes it and outputs it to the user. It also used a selection of sounds from the background to know when there is not one of these events happening, else there will false negatives.

One of our set ups to get a useful output and test the amplifiers were working properly.

All in all the project did actually work. It detected gunshots and screams being played into the microphone, and the triangulation algorithm worked, just not in real time. We managed to win the best masters project, mainly because we had good quality hardware, a partially working system and a good business case behind it. There is a lot of scope of where this project could go, and many things that could be improved, but we were happy with how it came out. I may be able to use some of the circuitry on other projects, who knows. If you are interested in more of the project, maybe some more detail about the hardware or manufacture, comment or message on Twitter. Thanks for reading.

A good example of how much difference there is between the microphones when a big sound was made. Minute distances made a big time difference.

Semi Autonomous Robotic Platform

As part of my degree I had to complete a project as part of the third year in the field of robotics and electronics. I chose to make a robotic platform, a simple idea that could be completed to a high quality with the right amount of effort. What is a robotic platform I hear you ask? well it essentially is a small buggy/rover that that moves around an assigned area completing simple jobs such as transporting goods, picking up parcels or any job that needs a moving vehicle. Usually autonomous, and very expensive, the majority of systems are very application specific. Some simple systems without any sort of control system can cost tens of thousands of pounds, and are not easy for the average employee to operate. Tackling the problem of expensive, application specific robotic platforms was the basis of my project.

4WD robotic platform
The Nexus 4 wheeled drive mecanum robot has an arduino based control system, and mecanum wheels, but will set you back $1500

Named the Semi Autonomous Robotic Platform, the idea was very simple, make a modular system, with building blocks that could be easily interchanged, and didn’t cost the world. These modules were things like motor controllers, sensors and power systems. If a user had a working platform built from this system, it would take minimal effort to swap out any of these modules to bigger motors or better sensors. This means a user can make a robot and only buy the bits they need, and even make their own modules, as long as they fit to the standard written as part of the project.

system block diagram
The initial block diagram of the system, showing how the modules can be controlled in hierarchy structure.

In most robotic systems, mainly to keep costs cheap, there is one controller that controls everything. This idea makes sense for small integrated systems that don’t need to change, but doesn’t really work when systems need to be dynamic. For instance, say you decide that your DC motors driving your robot aren’t giving you the control you want. You source some stepper motors, but this means completely changing the motor controller and therefore the software that controls it. Because one controller is in charge of everything, the software for the whole system needs to be re-written, and re-tested. That small change could have affected any of the other systems that that controller is in charge of. Make a change that breaks something important, you could set back a project weeks. This shows how painful a setup like this can be, especially when it starts to become a complex robot. Add on top of that the potential for computer intensive algorithms being used on the robot, like route planning or SLAM, and that controller suddenly has a lot to do. My system design separates these jobs out to a selection of individual controllers, such as a system specifically for motor control, or power systems. These controllers can deal with the nitty gritty hardware, and leave the master controller to orchestrate a higher level version of control.

Final Year Project
My design, near the end of the project, with the mecanum wheels, ultrasonic sensors and multiple controllers.

The added benefit of separating out all these jobs means that multiple engineers can work on the same robot, at the same time on different areas and not be worried about breaking the other person’s design. The system specification defines how the modules interact in terms of communication speeds/type, the way to alert other modules and how those communications are scheduled. The master controller (shown in the system block diagram in green) schedules all these communications and decides which modules need specific information. Warnings, control signals and user inputs are all calculated and scheduled, then communicated to and from the required modules. A power system doesn’t care that a user has pressed a button to scroll through an LCD screen, and the master controller means it doesn’t see it.

The above video shows how the robot moves with its mecanum wheels, and how it can easy move around environments. I will explain the more technical parts of the project in a later post, but this simple idea became a very heavy hardware based project, rather than the software project it started as. I learnt about mechanical design, PCB design and good techniques associated with electronic design. For these reasons, the robot won the “Best Project” award for 2017. Thank you to: Cubik Innovation for help with electronic design, and providing PCBs, VEX Robotics for donating the wheels, and Altium Designer for providing their electronic design software. I would not have been able to produce the robot I did without them.