Barret Schloerke || Maximize computing resources using future_promise() || RStudio

00:00 Introduction
01:45 Setting up a multisession using the future package
02:05 Simulation using two workers
04:14 Simulation using 10 workers
05:20 What happens when we run out of workers?
05:35 How Shiny handles future processes like promises
07:16 Introduction to future_promise()
07:45 Demo of the promises package
09:21 Setting the number of workers
10:40 Demo of processing without future_promise()
14:11 Wrapping a slow calculation in a future()
14:53 Demo of processing using Plumber
16:25 Considerations on the number of cores to use
17:21 What happens if we run out of workers?
19:44 Decrease in execution times using future_promise()

In an ideal situation, the number of available future workers (future::nbrOfFreeWorkers()) is always more than the number of future::future() jobs. However, if a future job is attempted when the number of free workers is 0, then future will block the current R session until one becomes available.

The advantage of using future_promise() over future::future() is that even if there aren’t future workers available, the future is scheduled to be done when workers become available via promises. In other words, future_promise() ensures the main R thread isn’t blocked when a future job is requested and can’t immediately perform the work (i.e., the number of jobs exceeds the number of workers).

[Nikvaylo]

08:49 It's a real WOW moment. :D I can't wait to use those functions. Thank you for the video.

[giuseppedejanlucido]

That's amazing! I want to try this combo as soon as possible! 🤩
It's a little bit tricky to understand how "future" works 🤔 actually, I can't figure out the real differences with "parallel" and "parallelly" packages and why they can't be used for shiny and plumber

[hassassin]

Does ist work with spatial dataframes/ sf-dataframes and sf operations like st_intersection?

[gacolitti]

How did you create the animations? Did you say Keynote?