Algorithm
At the beginning of my project, we used beamforming technology which is widely used in radar. This technology is theoretically supported very well but need considerable computation. If you want to know details about this algorithm, please go to Resource page and visit Luting Yang's project webpage.
Later on, with the help of Ricky Chen, we found that using cross correlation was a really practical alternative method to calculate the time difference. As I mentioned in Abstract, there are 64 microphones in the array, meaning there are 64 channels of audio signals. Each signal goes slightly behind or in advance against its adjacent ones. By calculating the cross correlation of two signals, we could get their time difference according to which we shift the signals afterwards. We choose one microphone as a fixed position and shift all other signals with regard to this position. Then all 64 shifted signals will be added together to increase the total SNR.
Compared with the original beamforming technology, our new algorithm has been proved to not only magnificently reduce the computation for time difference but also its accuracy. Using the time difference table we obtain from cross correlation, the shifting become much more precise and correspondingly the final SNR improvement much more approaches to the estimated level.
There is one disadvantage about our new algorithm compared with beamforming. Beamforming could allow us to have a comparatively accurate direction of a sound. Theoretically, we are also able to have the correct direction of a sound based on the time difference table by linear regression. However, currently the result is not very encouraging.
Later on, with the help of Ricky Chen, we found that using cross correlation was a really practical alternative method to calculate the time difference. As I mentioned in Abstract, there are 64 microphones in the array, meaning there are 64 channels of audio signals. Each signal goes slightly behind or in advance against its adjacent ones. By calculating the cross correlation of two signals, we could get their time difference according to which we shift the signals afterwards. We choose one microphone as a fixed position and shift all other signals with regard to this position. Then all 64 shifted signals will be added together to increase the total SNR.
Compared with the original beamforming technology, our new algorithm has been proved to not only magnificently reduce the computation for time difference but also its accuracy. Using the time difference table we obtain from cross correlation, the shifting become much more precise and correspondingly the final SNR improvement much more approaches to the estimated level.
There is one disadvantage about our new algorithm compared with beamforming. Beamforming could allow us to have a comparatively accurate direction of a sound. Theoretically, we are also able to have the correct direction of a sound based on the time difference table by linear regression. However, currently the result is not very encouraging.
Simulation
Matlab codes changes a lot during the whole project due the different algorithms we apply.
During the early stage, since the original beamforming technology need tremendous computation including 360-degree iteration and time-frequency-domain transformation, we developed a parallel-computing-like model. Choose a manager computer whose duty is to assign the heavy computational workloads to the worker computers, collect their results and do a little computation based on their results. Connect a certain number of worker computers to the previous manager computer so that they can receive raw data from the manager computer, focus on computing and send the segmental result back to the manager computer. Such mechanism makes real-time direction obtainment come true.
Later on, since new cross correlation algorithm was applied, computation cost was not a problem any more because there is no necessity to try 360 degrees to get time difference table. All the computation workload could be done on a single computer in real-time. Hence, I keep both algorithm in the folder for different uses.
Since most of my time was to prove the correctness of the beamforming technology and the idea that adding shifted waves could increase the SNR, getting the estimated direction of the sound source was my prime goal. Therefore, we still use beamforming technology to get the direction. The "manager.m", "worker.m", "manager_Realtime.m" and other relevant dependent files are for this purpose. Among them, "manager.m" is for simulation on Matlab and its sound source is a recorded binary file while "manager_Realtime.m" is for real-time simulation on board and its sound source is a movable speaker.
/********* For Correctness Simulation *********/
1. Make changes in "Configuration.m" according to your situation and run it.
2. If you choose to run your simulation on a single computer (isTeamwork = 0 in "configuration.m"), just run "manager.m" and ignore following steps.
3. If you choose to run your simulation within a network involving several computers (isTeamwork = 1 in "configuration.m"), follow the coming steps.
4. Pay attention to "IP&Port" part. Carefully modify your manager's IP.
5. Run "worker.m" first. If you need X worker computers, then run "worker.m" in Matlab on other X computers. If you use the parallel computing feature of Matlab, change "for" into "parfor" in "worker.m" and open that feature in advance on each work computer.
6. Run "manager.m" on the manager computer in Matlab.
/********* For Real-Time Test *********/
1. Make changes in "Configuration.m" according to your situation and run it.
2. You must choose to run your simulation within a network involving several computers (isTeamwork = 1 in "configuration.m"). The number of the worker computers involved depond on the performance of your computers. Highly suggest you to do computer based simulation first to know roughly the number of the worker computers you need.
3. Pay attention to "Mode" part in "Configuration.m". Modify it according to your situation.
4. Pay attention to "IP&Port" part in "Configuration.m". Carefully modify your manager's IP.
5. Run "worker.m" first. If you need X worker computers, then run "worker.m" in Matlab on other X computers. If you use the parallel computing feature of Matlab, change "for" into "parfor" in "worker.m" and open that feature in advance on each work computer.
6. If your upper microphone board is powered by FPGA board, open "MultipleDaqAcq.vi" on manager computer, click on start after all worker computer are waiting.
7. If your upper microphone board is powered by NI Elvis II platform, please open "MultipleDaqAcqElvis.vi" instead and click on start as in step 6.
After proving the correctness of the above ideas, we start to see the real effect of such ideas. Here our new cross correlation algorithm is applied. This test is separated into two sub tests, one source test and two source test.
/********* For one-source test *********/
1. place a speaker.
2. change MODE to 3 in "configuration.m"
3. see ReadMe.txt in LabView code folder to open a proper .vi file
4. run "audioPlay_RealTime.m" on LabView.
/********* For two-source test *********/
1. place a speaker and make a record.
2. change MODE to 4 in "configuration.m" and modify FILEID in "saveLagTable.m"
3. run "saveLagTable.m" on Matlab to store a lag table for this speaker.
4. introduce another speaker as an interfere source
5. see ReadMe.txt in LabView code folder to open a proper .vi file
6. run "audioPlay_RealTime.m" on LabView.
During the early stage, since the original beamforming technology need tremendous computation including 360-degree iteration and time-frequency-domain transformation, we developed a parallel-computing-like model. Choose a manager computer whose duty is to assign the heavy computational workloads to the worker computers, collect their results and do a little computation based on their results. Connect a certain number of worker computers to the previous manager computer so that they can receive raw data from the manager computer, focus on computing and send the segmental result back to the manager computer. Such mechanism makes real-time direction obtainment come true.
Later on, since new cross correlation algorithm was applied, computation cost was not a problem any more because there is no necessity to try 360 degrees to get time difference table. All the computation workload could be done on a single computer in real-time. Hence, I keep both algorithm in the folder for different uses.
Since most of my time was to prove the correctness of the beamforming technology and the idea that adding shifted waves could increase the SNR, getting the estimated direction of the sound source was my prime goal. Therefore, we still use beamforming technology to get the direction. The "manager.m", "worker.m", "manager_Realtime.m" and other relevant dependent files are for this purpose. Among them, "manager.m" is for simulation on Matlab and its sound source is a recorded binary file while "manager_Realtime.m" is for real-time simulation on board and its sound source is a movable speaker.
/********* For Correctness Simulation *********/
1. Make changes in "Configuration.m" according to your situation and run it.
2. If you choose to run your simulation on a single computer (isTeamwork = 0 in "configuration.m"), just run "manager.m" and ignore following steps.
3. If you choose to run your simulation within a network involving several computers (isTeamwork = 1 in "configuration.m"), follow the coming steps.
4. Pay attention to "IP&Port" part. Carefully modify your manager's IP.
5. Run "worker.m" first. If you need X worker computers, then run "worker.m" in Matlab on other X computers. If you use the parallel computing feature of Matlab, change "for" into "parfor" in "worker.m" and open that feature in advance on each work computer.
6. Run "manager.m" on the manager computer in Matlab.
/********* For Real-Time Test *********/
1. Make changes in "Configuration.m" according to your situation and run it.
2. You must choose to run your simulation within a network involving several computers (isTeamwork = 1 in "configuration.m"). The number of the worker computers involved depond on the performance of your computers. Highly suggest you to do computer based simulation first to know roughly the number of the worker computers you need.
3. Pay attention to "Mode" part in "Configuration.m". Modify it according to your situation.
4. Pay attention to "IP&Port" part in "Configuration.m". Carefully modify your manager's IP.
5. Run "worker.m" first. If you need X worker computers, then run "worker.m" in Matlab on other X computers. If you use the parallel computing feature of Matlab, change "for" into "parfor" in "worker.m" and open that feature in advance on each work computer.
6. If your upper microphone board is powered by FPGA board, open "MultipleDaqAcq.vi" on manager computer, click on start after all worker computer are waiting.
7. If your upper microphone board is powered by NI Elvis II platform, please open "MultipleDaqAcqElvis.vi" instead and click on start as in step 6.
After proving the correctness of the above ideas, we start to see the real effect of such ideas. Here our new cross correlation algorithm is applied. This test is separated into two sub tests, one source test and two source test.
/********* For one-source test *********/
1. place a speaker.
2. change MODE to 3 in "configuration.m"
3. see ReadMe.txt in LabView code folder to open a proper .vi file
4. run "audioPlay_RealTime.m" on LabView.
/********* For two-source test *********/
1. place a speaker and make a record.
2. change MODE to 4 in "configuration.m" and modify FILEID in "saveLagTable.m"
3. run "saveLagTable.m" on Matlab to store a lag table for this speaker.
4. introduce another speaker as an interfere source
5. see ReadMe.txt in LabView code folder to open a proper .vi file
6. run "audioPlay_RealTime.m" on LabView.