IDEA

We approached a small distillery with the idea to use FaceTime to promote there unique brand. FaceTime had never been used in this way before so the project would be worlds first.

CHALLENGES

We would need to create a device that could recognise an incoming FaceTime call on an iPad and answer the call. We would then need to create a video that would auto play at the right time and be responsive to the users voice commands. We also wanted to record responses from the user, so we would need another device that could record the screen of the iPad.

SOLUTION

I built a bit of software that used color recognition to recognise the incoming call and trigger a robot controlled by an Arduino to actually physically push the call answer button on the iPad. This same piece of software was used to trigger a video to play on a monitor placed in front of the iPad to display to the user. I then connected up the speaker from the iPad to a microphone that listened at the right time for the users voice when asked what unique batch number they had. The response was fed back into google's free voice to text translator and the output used to control and display the correct video's batch number. To record the users response another piece of software was set and triggered at the correct time to record the iPad's screen and therefore record the users message at the end of the call.

TECHNOLOGIES USED

ACTIONSCRIPT, CSS, HTML5, JAVASCRIPT, ARDUINO

RESULT

The system was set up and run for a year, within that time it was used by over 200 people and the project was successful in winning a BIMA in 2015. Below is a video that shows the Award Entry video that was submitted to show the complete process from concept to final product.

Below is a cut-down video that shows what the user experienced when they FaceTime called Tarquin.

Behind the scenes look at all the parts needed to put it together.