filmov
tv
Uncluster Your Data Science Using Vaex • Maarten Breddels & Jovan Veljanoski • GOTO 2021
Показать описание
This presentation was recorded at GOTO Copenhagen 2021. #GOTOcon #GOTOcph
ABSTRACT
Would you like to build an snappy dashboard visualising hundreds of millions of data points, or interactively explore hundreds of Gigabytes of data, all of that using a single machine?
Meet Vaex - an out of core DataFrame library in Python that can do all the typical data manipulations, filtering, and aggregations on a billion rows in real time & on a single computer. This approach empowers your team and allows them to focus much more on the business problem, as it removes the large DevOps overhead of configuring and maintaining a cluster.
Vaex fully supports Apache Arrow, which both facilitates the interoperability with other systems and enables storage and manipulation of more complex data structures like lists [...]
TIMECODES
00:00 Intro
00:50 Motivation
05:20 Vaex
06:14 Concepts: Memory mapping
07:56 Concepts: Column based storage
09:37 Concepts: No memory copies
10:50 Concepts: Compute & expression system
13:30 Demo
32:32 In production
34:23 In the wild
35:00 In production: Dash example
37:13 Summary
37:58 Outro
Download slides and read the full abstract here:
RECOMMENDED BOOKS
#Vaex #ApacheArrow #DataScience #AI #ML #ArtificialIntelligence #MachineLearning #DataFrame #Programming #VaexIO #Astronomy
Looking for a unique learning experience?
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
ABSTRACT
Would you like to build an snappy dashboard visualising hundreds of millions of data points, or interactively explore hundreds of Gigabytes of data, all of that using a single machine?
Meet Vaex - an out of core DataFrame library in Python that can do all the typical data manipulations, filtering, and aggregations on a billion rows in real time & on a single computer. This approach empowers your team and allows them to focus much more on the business problem, as it removes the large DevOps overhead of configuring and maintaining a cluster.
Vaex fully supports Apache Arrow, which both facilitates the interoperability with other systems and enables storage and manipulation of more complex data structures like lists [...]
TIMECODES
00:00 Intro
00:50 Motivation
05:20 Vaex
06:14 Concepts: Memory mapping
07:56 Concepts: Column based storage
09:37 Concepts: No memory copies
10:50 Concepts: Compute & expression system
13:30 Demo
32:32 In production
34:23 In the wild
35:00 In production: Dash example
37:13 Summary
37:58 Outro
Download slides and read the full abstract here:
RECOMMENDED BOOKS
#Vaex #ApacheArrow #DataScience #AI #ML #ArtificialIntelligence #MachineLearning #DataFrame #Programming #VaexIO #Astronomy
Looking for a unique learning experience?
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
0:38:25
Uncluster Your Data Science Using Vaex • Maarten Breddels & Jovan Veljanoski • GOTO 2021
0:38:28
Tom Augspurger: Scalable Machine Learning with Dask | PyData New York 2019
0:13:47
Machine Learning, Data Science and Deep Learning with Python: Presenting your Data
0:09:51
Cluster Analysis in just 10 Minutes| Clustering tutorial for Beginners| Data Science with ExcelR
0:25:18
Arkouda: Terascale Data Science at Interactive Rates |SciPy 2020| Ben Albrecht
0:50:59
Unlocking the True Value of Big Data with Open Data Science
0:28:52
Scalable & Deployable Data Science with Anaconda | Kris Overholt | AnacondaCON 2017
0:45:11
Raoul Gabriel Urma & Valentin Dalibard - Interactively Analyse 100GB of Data
0:06:57
Apache Spark Explained: Key Features and Why You Should Learn It!
1:14:50
Fireside chat #12: Kubernetes for Data Scientists
1:34:12
Data Science Crash Course
0:34:49
Aaron Richter: Your data fits in RAM: How to avoid cluster computing | PyData Miami 2019
0:49:27
Introduction to scalable computing with Dask in Python
0:54:59
So you want to build a Greenplum?
1:02:00
Kaggle Live Coding: Identifying the most important words in a cluster | Kaggle
0:35:28
Marck Vaisman | Scaling up to Big Data Devops for Data Science
1:45:03
Enterprise Data Science at Scale Meetup Oct 2017
0:26:01
Patterns for Successful Data Science Projects (DataEDGE 2018)
0:29:52
CERN’s Next Generation Data Analysis Platform with Apache Spark with Enric Tejedor (CERN)
0:17:32
Federated Analytics: Using Nuix and PXF to Load Unstructured Data into Greenplum
0:12:02
'cluster Analysis' : Distance B/w Clusters | Case Studies Of Clusters | Data Science [2018...
0:13:38
Operationalization of ML Model Using Azure ML and Kubernetis Cluster [Part 4/4]
0:02:36
Vaex: Get Summary Statistics of a Vaex Dataframe
1:58:26
Managing your AI and Data assets in the enterprise - A hands-on workshop
Комментарии