Deep Learning Program Hallucinates Videos | Two Minute Papers #120

The paper "Generating Videos with Scene Dynamics" and its source code, and a pre-trained network is available here:

Recommended for you:

WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE:
Sunil Kim, Daniel John Benton, Dave Rushton-Smith, Benjamin Kang.

Károly Zsolnai-Fehér's links:

[jamiealkire]

These breakthroughs are incredible. I feel like we're witnessing the work of literal wizards. What stupendous progress!

[kipper]

3:39 Could we potentially use this "future prediction from a still image" to create an effect like the moving pictures from Harry potter on old photos?

[kipper]

2:40 Look in golf at the two videos in the lower-left corner.

That guy in brown is reaching into the video next to him!

[nBoxes]

Yay! One more work that has publicly available code on the website.

[thelethalmoo]

I think this is my new favorite channel! Makes me really look forward to the future!

[adamvitkovsky]

Finally!!!
I envisioned this system the first time you presented the network that can generate pictures just from a crude description.
What I envisioned was a Hollywood movie maker system where a director would just describe a perfect scene and the AI would “dream it up” to the last detail, the director could then say I’d rather have it in a rain or in sunset, same would go for main actors and the whole cast a “I need a bit taller, blond a bit Audrey Hepburn like” etc...
And sure enough we already have the celebrity face hi-res generator network, so I reckon in 10 years we will watch first such a dreamed up blockbuster.
But we may find this technique used earlier in video games where the whole game environment and characters would be “dreamed up” it’s just the quality wouldn’t be quite movies CGI level.

[halnineooo]

We are not measuring the impact of upcoming artificial vision. Beyond technology this will change how science itself is made. One step closer to AGI.

[chkone]

Usually I'm suspicious when we use "perceptual metrics". I remember a comparison: MP3 vs uncompressed sound. The young people prefer MP3 and older people prefer uncompressed sound.
I know on many cases the perceptual metric is the only one available, but I just don't like it (even if I use it somethime).
That could be tricky may be on the future people will prefer compressed video or JPEG artefact. Or when you see a movie at 60 or 120 Hz (compared to 27-29 Hz Standard) that look weird even if it is objectively better :) (independently of the artistic consideration)

[knoriy]

I've been watching his videos for a couple of days and find myself hooked. Also his names is cool.

[tothesun]

I want to see these methods used in sound design. Have the network listen to a huge number of different instrument or effect presets and then tell it to come up with original sounds no one's ever heard before.

[MrtinVarela]

Great, someone found the source code for the programs used in police action movies/series.

[Grayhamper]

yay, another Five Minute Papers video!

[RaduOleniuc]

Lunch time chat on 2014/05/12:

a computer vision scientist: How long does it take to train these generic features on ImageNet?
Hossein: 2 weeks
Ali: almost 3 weeks depending on the hardware
the computer vision scientist: hmmmm...
Stefan: Well, you have to compare the three weeks to the last 40 years of computer vision

[SweetHyunho]

Public video training should be made a open-source distributed computing project. That would yield a great public resource.

[Kram]

Neat! I was hoping for this. Still ways off but it's looking promising.

[ruagrill]

i wonder if this technique could be modified to do interpolation instead of extrapolation
as in you feed it an initial image, and a second image of the same scene taken say 10 seconds later, and it tries to generate a plausible sequence of frames in between

[smmoom]

i wonder if this same idea would work for audio and speech synthesis!

[petersmythe]

I think we could genuinely see single player video games in a decade or so that have no independent engine, lighting, models, physics, etc. Just neural networks trained to respond in the appropriate fashion to user input and generate output that like a game. While the training process would be very complex and expensive, the actual game could probably run smoothly on very weak hardware because there isn't anything inside, just random seed, previous frames, any saved data, and user input.

And multiplayer with almost no network load is also quite possible, with some obvious caveats that there should be at least a somewhat consistent reality between both worlds.

[Rolyataylor]

There is enough Simpsons for a neural network to use. It would be cool if you could feed a script to a neural network and it would generate Simpsons episodes.

[selfreference]

These are my favorite kind of video. :)