Using audio for open-source intelligence analysis
Introduction
Open-source intelligence analysis often heavily relies on video analysis. There will be use cases for researchers to look at video footage to find people, buildings or other things that will be a form of verification and or validation based on analyzing video footage using OSINT techniques.
In most cases these techniques rely on image analysis techniques. OSINT researches study video footage and look for details in:
The foreground
The background
Look for distinct makers
Metadata (exif)
Who posted this (5w1h)
People rarely analyze the audio of video footage. Although this could provide the OSINT investigator with new clues and pivot points that they may want to corroborate within their overall analysis.
This article will discuss methodologies and tools that can be used to analyze audio. This will not be limited to analyzing audio that is part of video footage because audio is everywhere around us all over the world
Where to find the audio
First of all it is good to understand where (online) audio can be found. It is good to understand that audio basically can be found anywhere online. Secondly an OSINT investigator may obtain or get handed (client seed data) one or more audio files that are common from (semi)closed sources for example a forensic copy of a smartphone or someone sharing a video or audio file from a closed social media group.
Common audio sources:
TV stations (online)
Radio stations (online)
Streaming services (social)
Podcasts
Videos
Audio clips
Social media posts
You have obtained an audio file, now what?
Once the investigator has a need to analyze the audio most likely they will have a goal WHY they need to analyze the audio. Some examples of goals may be:
Listening for people's names (full names, nicknames etc.)
Listening for location names (country, city, street, postal codes etc.)
Listening for other names (Business, venue, restaurant, conflict area etc.)
Listening for identifiers related to people (family members, email, phone number, school names etc.)
Listening for area specific sounds (Traffic light tickers, airport, public transport, specific animal sounds, weather, tv/radio/music in the background, language/dialects)
Understanding who talks to who (hierarchy)
Understanding the narrative (5w1h)
Understanding the when (time/date)
Understanding where
Understanding why
Example audio file:
Now close your eyes. Listen to the audio. What do you hear?
People talking
How many?
What Language(s)
What are they saying
Rumble sound
What is making that rumble sound?
A bell
What kind of bell is this?
Back ground noise
What can I tell about what I am hearing?
Tools
General audio tools
Spoken Language identification
Speech to text tools
A tradecraft technique is to utilize the live audio translating tools that most Search Engine providers have available for mobile devices. Simply point your mic to the audio (recording) with using any of the below apps and the app will live translate what it can hear.
Google translate (mobile) < turn on mic listener
Bing translate (mobile) < turn on mic listener
Yandex translate (mobile) < turn on mic listener
Apple translate (mobile) < turn on mic listener
Alternatives that can be used in a Web Browser:
IBM Watson Speech to text (Lite offers 500 free minutes per month)
The Bear File Converter (Speech to text)
360 Converter (Multiple option to convert video, audio, speech and more into text)
Virtualspeech (Speech to text)
Verifying and validation of specific sounds
Animals
Nature
Urban
Sounds cities map (audio recording from cities all over the world)
Sound Cartography (various maps with various sounds from all over the world
Languages of the world
Omniglot languages of the world (listen to languages all over the world)
Spoken language identifier (helps determine what language is spoken)
Local Lingual (world map with languages)
Misc
British Sound Library (Listen to a selection from the British Library’s extensive collections of unique sound recordings, which come from all over the world and cover the entire range of recorded sound: music, drama and literature, oral history, wildlife and environmental sounds.)
And that's it for this audio OSINT blog. Feel free to share this blog with your community.
And remember some of us can see sound ;-p
