Traditional sampling techniques (grid vs random vs sobol vs latin hypercube)

Welcome to video #1 of the Adaptive Experimentation series, presented by graduate student Sterling Baird @sterling-baird at the 18th IEEE Conference on eScience in Salt Lake City, UT (Oct 10-14, 2022). In this video, Sterling introduces the concept of adaptive experimentation and covers traditional sampling approaches, including grid, random, Latin hypercube, and Sobol sampling. He also discusses the use of discrepancy as a metric for performance evaluation. Don't miss the next installment in this informative series on experimental optimization.

0:00 introduction to adaptive experimentation
2:09 Comparing grid/random search with quasi-random search with adaptive experimentation approaches (grid vs human intuition)
4:40 traditional optimization jupyter notebook tutorial
6:14 grid sampling
7:45 latin hypercube and sobol sampling
9:36 comparing different sampling
11:05 discrepancy comparison in low and high dimensional data

[christophchristoph]

Awesome series! Thank you so much for this!
One thing that came to mind while watching this video:

The quasi-random methods presented assume a continuous space of independent variables. 
If you have a discrete set of materials (e.g., a set of recipes created according to a particular subject logic), these assumptions could be violated. This could be the case if the amount of one ingredient is in a certain ratio to another, the materials have different densities, and the materials are described by a finite volume.
Here, random selection has the advantage of not making any assumptions. To minimize the discrepancy, one could use k-means.

[mattmiller]

Wonder how well sampling based on space-filling curves (like Hilbert’s) could perform. Great discussion, have always been interested in such measures (like discrepancy). 🤗

[blackspell]

Hi Taylor, great video! Can you provide the figure of minute 12:40? the jupiter notebook doesnt work for me :/

[probinertasks]

I was looking for a comparison between Sobol and BRJ (Basic Random Jump). I did not know about Latin Hypercube. Do you have any insight on BRJ, ever looked into it? Cheers

[nano]

But what's the point in not evenly distributing your data points? If you're searching for effects then shouldn't you be doing that?