Scientists Use AI to Translate the Language of Whales

A marine biologist reacts to the artificial intelligence being used to study the language of whales and dolphins. Can scientists use AI to communicate with animals?

00:00 - Can Artificial Intelligence Translate the Language of Whales?
01:12 - How Do Whales Communicate?
03:34 - Conversation With a Humpback Whale
05:35 - How AI Is Translating Dolphin Language
06:37 - Whale Phonetic Alphabet
08:31 - Negative Impacts of Artificial Intelligence

In Sweden, scientists from the Royal Institute of Technology and Kolmarden Wildlife Park identified what might be a dolphin’s laughter. Off the island of Dominica, an international initiative called Project Ceti used advances in AI to reveal intricate structures within whale communication in what some are calling the whale phonetic alphabet. And in an unprecedented encounter, a research team from UC Davis and the Alaska Whale Foundation successfully engaged in a 20 minute “conversation” with a humpback whale. Are humans on the verge of interspecies communication with whales?

It’s important to note that when we talk about whales, we’re talking about several unique species including dolphins and porpoises. All whales, dolphins, and porpoises are classified as whales in the Cetacea order. Within the Cetacea order are two suborders: Mysticeti and Odontoceti. These suborders have very different vocalizations and means of producing sound. Odontoceti are toothed whales like beluga whales, orcas, and dolphins who produce rapid bursts of high-frequency clicks, whistles, and pulses by passing air through a structure in their blowhole called the phonic lips. These sounds are then focused through their melon. Mysticeti (baleen whales like humpback whales) do not have melons and phonetic lips. Instead, they have a larynx with vocal folds. But they don’t have to exhale in order to produce sound. Rather, they appear to capture air in a laryngeal sac and recycle it back to their lungs.

Original Videos

Type D Killer Whale Research Team 2019,
Taken under Chilean Sub-Secretary of Fisheries and Aquaculture
Research Permit, REs. Ex. 1811 (31 May 2017) and Res. Ex. 4402 (18 December 2018)

Humpback Whale Vocalizations
Research conducted under permit NOAA/NMFS 19703

Sources

Join KPassionate:
----------------------------
Check out our Patreon!
--------------------------
Social media:
-----------------------
Music

Additional Images
Voice of America
NOAA Fisheries

#whale #AI #kpassionate

[KPassionate]

Learn more about these incredible animals!

[carlharrison]

When the Whales start saying "stop dumping that crap in my house" - you know its been cracked.

[amateurmakingmistakes]

About 15 years ago, I went for a surf and when I got to the beach, there was a huge crowd. Thankfully, it wasn't a shark attack or a drowning, but a large whale (humpback or sperm whale?) had been in the area for a few hours, staying mostly in the same spot. After taking photos for half an hour, I went in the water. I was probably about 70 metres away from the whale and it was obvious it was checking out surfers. Twice, I heard guys ask if the whale might attack. I just thought that if it wanted to do that, it would've done so by now. No, it was really grokking on our species.

Knowing that they're very intelligent and have excellent hearing, I slapped the water as hard as I could. Once. The whale then hit the water with a fin, once. I then hit the water twice, leaving a second or so in between slaps. The whale then slapped its fin twice. Yeah, I then hit the water three times and the whale responded by slapping its fin three times. No further submissions, Your Honour, but it sure as hell seemed like this beautiful animal wanted to communicate. After that, a man let his teenage daughter approach a bit closer. The whale stayed there long enough for me to shoot more photos for half an hour afterwards. Small surf, but one of the best days in my nearly 60 years of surfing.

[zipperpillow]

2 ravens led me to a freshly trained-killed moose. After I opened it up and cut off its hind quarters, the ravens could feast on the guts. Birds can communicate with people if you bother to learn their lingo.

[scraller]

The dolphins are saying, 'So long and thanks for all the fish' 😄 Another excellent and thought provoking video, enjoyed it!

[Thomas-mjdv]

If the only reason we created AI was to be able to talk to other animals, then it was worth it, great stuff )))

[findJLF]

The analogy to Freddie working the crowd with his interactions... was spot on. A classic media mis-representation of the actual experiment. Very interesting research and a well presented video. Completely cutting edge stuff... I can't wait to hear what we can understand of Whales and their communication. Eventually, we may find out that Whales are smarter than... most of us.

[nyyotam]

If whales have a spoken language, I hope someone teaches them to defend themselves as a group against whalers.

[GSBarlev]

I read the UC Davis paper when it came out. I'm equally parts amused, fascinated and *mortified* that we successfully carried out a conversation with an arguably sapient species... and the content amounted to essentially a *crank call, * with the humpback ending the convo by "giving us the flipper." 💀

[GunnarCreutz]

Nice to hear that AI has been used to analyse sounds from whales instead of just creating bullsh*t.

[dianacryer]

I can really appreciate how a clip of Freddy was woven into a video about whales.
I love it.

[spyrlblade]

The interaction with Twain seems like seeing a cat, and meowing back at it every time it does.

[therealzahyra]

5:05 It's like we said "Hi" and the other whale said "hi" back and we just kept saying "hi" to each other 😂

[eruiluvatar]

As an AI developer (for a small company, I don't work for any of the leading ones) I believe that there is a good chance of being able to build an AI that can translate animal vocalizations if they actually form a language even if it is extremely different from ours. The path towards that could look like this:

- We first need to figure out the words/syllables/alphabet (so we can tokenize the inputs), not the meaning, just the individual sounds. And that seems like what they are doing right now.
- Once we have words, we can either train an autoencoder or an autoregressive next word predictor with a bottleneck in the middle. That on its own won't give us a translation (may give us a chatbot that only wales understand) but if successfully trained, we can take the activations of the artificial neurons in that bottleneck and use those as an embedding (turns each word into a many dimensional vector, when we do that for human languages individual numbers of that vector can represent things like meaning, ie "dog" and "cat" will share quite a bit of values with each other and with "animal"). We won't know what does each dimension of that embedding mean.
- Once we can tokenize and embed we can train an LLM. If we just do that, we would just end up with a better chatbot that only whales understand and that only knows what whales know. (we could do that and let it talk with whales to try getting a bigger dataset)
- But we can try something else: Instead of train a whale LLM from scratch, we add another two dimensions to the whale embedding and make sure that the number of dimensions of the embedding that we have generated matches the ones we generate for humans. Those two extra dimensions would contain an 1.0 in only one of them depending on whether it is whale or human language. Then we take a pre trained LLM for humans with those two dimensions added (or two pre existing ones but unused or barely used, repurposed) and fine tune / continue training with both the whale dataset and the human dataset alternating samples of each in each batch.

The last part is real tricky but even if the meaning for each dimension excepting the two that we introduced by hand won't match at all, forcing a transformer to understand both under the same architecture is likely to lead to some unified internal representations at the outputs of the intermediate layers (backpropagation and gradient descent would in some sense try to reuse what is in there from the human pre training, interleaving with human data would prevent progressively deviating from that too much). The transformer architecture used in modern chat bots would be particularly apt with this as it acts as some kind of progressive refining through algebraic operations between the embeddings of the tokens (that can act as logical reasoning) in the attention operators and each layer MLP can look up extra related meanings related to the input or do some non linear operations, the fact that the MLPs don't have outputs larger than the embedding space forces the network to compress the reasoning for the next layers in a single embedding vector. That is likely to indeed force some kind of internal translation for at least some of what was learned.

It is unlikely to work perfectly but I'd expect that if you train such chatbot and you just ask "What does this mean in English?" followed by some whale language. It would likely answer something vaguely related to what the whale intended, at least some times. That if it works at least to some extent could be an stepping stone that could be iterated upon. There is likely an optimized topology and training procedure that would maximize the reuse of features learned from human languages.

Once we have even rudimentary communication, that could bootstrap the building of more comprehensive datasets. We could just ask the whales to ELI5 when we don't understand lol. They may even be interested in helping us understand, something like we show them something = they tell us the whale word for that.

I hope someone does that and that it works, it would be about the closest thing to meeting intelligent aliens and good practice for when that may happen too.

[John_Weiss]

Always love it when someone provides their sources!

[gildedbear]

Thank you for this. The reports of "scientists had a 20 minute conversation with a whale" have bothered me because it's not a conversation if you don't know what you're saying (it's okay to not know what /they/ are saying because that's how both of you learn what the other is saying). So knowing that it was repeated contact calls that were returned, with matching intervals, makes me believe that we were basically saying, "hey. Hey. heeey. hEeeeey." and the whale was responding in kind.

It makes me think that some researchers should synthesize their own contact calls and play them each time they approach the pods they are studying. (specifically contact calls for the BOAT) We won't know what the "name" /means/ (if anything) but it would be very interesting to see if the whales start reacting to that particular contact call.

[b.a.erlebacher]

Most people don't realise that most human languages are nothing like each other, if they haven't been related for only some few thousand years. Parts of speech we're familiar with can't be expected to be the way innumerable human languages work. An animal language can't be expected to resemble a type used by human language groups spoken by billions, nor even the many many thousands working in some completely different way, plus even a larger number that hasn't been anybody's natural language for just a few generations. Most human languages in different groups are not at all like each other, and you can't expect an animal language, even if fairly large or meaningful, to resemble a common human one.

[CIONAODMcGRATH]

I would argue that music DOES contain "information".

I agree with the criticism of describing complex non-human communication systems with an anthropocentric focus.

Fascinating video! Thanks for the insights.

[johnminehan]

"Hey, thanks for all the fish. Hey, human, why are you laughing . . . ."

[BiddyBiddyBiddy]

Fantastic video, packed with info and I love how you explain everything beyond just "they had a conversation." I subscribed.