How To Setup Your Development Environment In Windows Using WSL For Airbyte

We didn't forget about you folks on Windows! By the end of this video, you'll have a complete dev environment and be able to move data with Airbyte in no time!

Commands to run:

Installing wsl:
wsl --install

sudo apt update && sudo apt upgrade

Installing Git:

sudo apt-get install git

For installing Python 3 in WSL
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
apt list | grep python3.10
sudo apt install python3.10
sudo apt install python3.10-distutils
sudo apt install python3.10-venv

Resources:

[dn]

Mini troubleshooter for some things you might run into:

1. After installing new version of python, the command "python3 --version" still prints the previous version
"python3" is just a symlink which points a specific python version. Like a shortcut.
By installing a new python version, you need to change that symlink to point to the new python binary (in my case, python3.11).

You can do that by running the following commands in wsl:

1. "sudo unlink python3" - this unlinks the symlink. Try typing "python3 --version". It's not found
2. "sudo ln -s <PATH_TO_PYTHON_BINARY>" - this creates a symlink with the name "python3" (the one we removed before) and points it to the newly installed python vesion

<PATH_TO_PYTHON_BINARY> is of course the path to the newly installed python. You can find it easily by running "which pythonX.x", where x.x is the version you want to locate, for example "which python3.11".

2. Docker Desktop complains about not having hardware assisted virtualization & data execution protection enabled in BIOS
launch cmd as admin and run the command "bcdedit /set hypervisorlaunchtype auto". Then restart the computer.
Note this may not solve the issue, as hyper-v must be installed. In my case, it was installed but not configured correctly for docker to run.

3. Docker Desktop -> Config -> Resources -> WSL Integration -> You cannot find your wsl in order to select it
make sure you have installed wsl2 in your computer
then, make sure that wsl version 2 is enabled for that specific wsl distribution you want to integrate with (for example Ubuntu 18.04) by running in PowerShell "wsl --list --verbose" and checking the VERSION column to be "2".
if the column prints "1" then copy the name of the distribution (you can find this in the output of the previous command, in the NAME column), for example "Ubuntu-18.04" and run this command in PowerShell "wsl --set-version <DISTRIBUTION_NAME> 2". (don't forget the "2" in the end, it's not a typo!)
the above process might take a few minutes. After this, no need to restart, just go to docker desktop and click the "Refresh" button a few times until your distribution shows up.

4. Running in wsl "docker-compose up" returns "no configuration file provided: not found"
either you need to clone the git repo in order to obtain the docker image which contains airbyte, or you need to cd to the directory where the repo is cloned.
cd into the directory which you cloned the repo and run "sudo docker-compose up"

[edgodalle]

Thanks for the vid. It is missing the part of cloning the github repo of airbyte. That part is present on the Mac version though

[Lapookie]

Please do an update on how to install octavia CLI for windows/wsl. It does not work with version > 0.44

[pbbbbbbbb]

I have followed the instructions, cloned the directory and cd into it but when I run sudo docker-compose up I get the message "no configuration file provided: not found". Any ideas how to resolve?

[srh]

Did you forget to clone Airbyte Git repo?

[Zombiezzz]

Really great video but I have a few questions. I was able to run Airbyte by just using Windows Powershell and not using Ubuntu. So what is the advantage of using Ubuntu when I already have python along with all my files in my Windows environment with Windows Powershell? Also, if I use the Windows Powershell, am I running docker/Airbyte in a Windows environment or Linux? I am still new to learning about this. Thanks!

[kingsabru]

Thanks a lot for this video. It's well broken down!
I have followed your instructions and have been able to setup Airbyte on Windows. My current challenge is with connecting to a localhost postgres database. Any chance of creating a demo for that?

[gustavoleo]

Nice tutorial! Tks! Will be great if could make a one or maybe a blog post with images regarding the impossible mission to find files if you choose to save result localy like, duckdb or csv... T

[lailaribke]

Hi, I just followed the video and the python version appears as Python 3.8.10. Al steps were completed successfully..

[dolamuoludare]

Hi, I just watched the video
Is the setup for airbyte?

[KevinConseil]

Great Video :) Thanks a lot!

Everything went fine until the step of "cd Development/" where I get the following message "-bash: cd: Development: No such file or directory"

Any idea how to solve this ?

Troubleshoot done: Re-run the following steps:
- sudo apt-get install git
- git config --global user.name "Your Username"
- git config --global user.email "Your Email"