Automating a Linux in Windows Dev Setup

Last time I used Windows on my main development machine was sometime in the early 2000s. Over the years, I had several side Windows machines (usually because I needed Visual Studio) but I was never really serious about using a PC as my main dev machine. Things have changed a lot in almost 20 years, and especially recently. Apple is moving to ARM and to everybody’s shock the Windows team has been investing in letting us run Linux distributions “natively” for a few years now!

Microsoft used to have an extremely negative view of Open Source but under a new leadership things have changed and they are making some drastic changes to be more welcoming to FOSS developers.

Could Windows provide me with a similar or maybe even better developer experience than macOs?

There is only way to know: I need to a Windows machine as my main driver. To be clear, I’m not trying to confirm/deny that Windows is becoming a better dev solution for developers in general. I’m only focusing on my own case: an open source developer coding using open source languages such as TypeScript, Go, Python, Rust and years of experience using Unix/Linux.

The first thing I wanted to try was to automate the setup of my Windows and Linux machine. I’ve been spoiled by Docker for years and I love reproducible environments. Here is my journey automating the set up of my Windows/linux dev environment.

Installing Linux/Ubuntu on Windows

The idea of having to use the Command Prompt or Regexit is reminding me of a distant past full of pain and anger. Let’s start by making this place a familiar home and install Ubuntu (Linux). Turns out you can install Linux from the app store thanks to a Windows 10 feature called the Windows Subsystem for Linux (WSL). The sales pitch is that you can run a GNU/Linux environment, including most command-line tools, utilities, and applications directly on Windows, unmodified, without the overhead of a traditional virtual machine or dualboot setup. You pick your Linux distribution from the Microsoft store, install it, and boom it runs locally. You can even invoke Windows applications using a Unix-like command-line shell and invoke GNU/Linux applications from Windows (you can even mix and match which is pretty crazy).

Bash command to copy the SSH key to the Windows clipboard

$ cat ~/.ssh/id_rsa.pub | clip.exe

For Machine Learning devs, there’s an extra good news: you can leverage your machine GPU from the Linux side (how many times did I wish I could use a GPU on my Mac?) WSL has been around for a little while and Microsoft recently released WSL2 to address some performance issues (and give Linux programs access to the GPU).

Enabling Windows Subsystem for Linux

To enable WSL2, you have to be on Windows 10 version 2004 (20H1). I know, the naming is super confusing, this isn’t the version from the year 2004 but the free update that was released last May. Windows stopped giving brand new names to updates. A bit like Mac OS X/MacOS but without the cute cat and location names. Since I was on this latest version, installing WSL and Ubuntu was trivial and took a few minutes. I just followed this very straightforward tutorial.

1. Launch PowerShell as Administrator (click the start menu, type PowerShell, expand the options and pick run as Administrator).
2. Type dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
3. Type dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
4. Restart your machine manually

Installing Ubuntu

At this point, WSL2 is setup, and we need to pick a Linux flavor (aka distro or distribution). I picked Ubuntu from the Microsoft Store (it’s free and you don’t need to signup or anything).

Ubuntu installed automatically on my machine in a few seconds and I got a prompt to set my linux username/password and I was done.

All in all, it probably took me less than 10 minutes, I’m impressed! Furthermore, I really like that I can run multiple versions of Linux while maintaining my main OS very stable. I did mess up my mac stability a few time when installing dependencies for some of the software I was developing. Also, it’s nice to be on a true Linux system, the same as the one I deploy to and it’s much lighter/easier than having to run everything in Docker.

Tips

You might want to configure how much resource you allocate to WSL if you feel Windows doesn’t get enough resources (you can also shut down the linux instance(s)).

You can export/import WSL distributions by using the wsl command from the windows side:

In Command Prompt, start by creating the folders we need

> mkdir %HOMEPATH%\Data\wsl\exports %HOMEPATH%\Data\wsl\distros

Then export the target Distro (here called Ubuntu)

> wsl --export Ubuntu %HOMEPATH%/Data/wsl/exports/ubuntu_%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%.tar

This created a distribution dump dated with today’s date.

We can then import it to create a new distribution/instance.

> wsl --import UbuntuReloaded %HOMEPATH%/Data/wsl/distros/ubuntuReloaded %HOMEPATH%/Data/wsl/exports/ubuntu_20200730.tar

The new distribution is called UbuntuReloaded and is a copy of the first instance. This is practical if you want to run a distro for Python and one for JS for instance. It’s also a nice way to backup an environment.

You can see the current distributions and their status by using:

> wsl -l -v

Note that exporting the distro requires shutting down the distro automatically, so save your work before running this command.

Note also that when importing an exported distro, wsl logs you in as root instead of the created admin user (which is obviously there).

Tooling & scripting

One of the many reasons I was interested in doing all my development on Linux is that I deploy my production code on Linux machines, so might as well develop on the same platform I deploy to. I’m also super familiar with Unix/Linux, not so much with Windows. And while I can run code via the command prompt or PowerShell, I really prefer not to and I miss my good old Terminal.

Terminal

Turns out Microsoft developed a great Terminal for Windows (and they open sourced it)! It just doesn’t ship with Windows but you can install it directly from the store.

Installation script

Besides Terminal, I need a few other tools, drivers and software to make my Windows machine truly useable. And like a lot of developers I like automating things. Partly because I’m a bit lazy but mainly because I hate to repeat the same task over and over. So I decided to write a script to set up the Windows and Linux side of things.

The idea is simple, I want to automate everything from the Linux side and for that I’ll use a bash script that I will backup on github as a gist so I will eventually be able to pull it down and run it from a new machine. The bash script is not designed to be generic but instead focuses on my machine with my dev env, my Windows apps, an option to install the device apps/drivers I might need.

dot files

My Linux config files (dotfiles) are shared between my machines using a tool called chezmoi and GitHub. I’m still planning on using my Mac and various other machines so managing my configs across machines makes a lot of sense.

Editor

I switched my editor to VSCode a long time and have been using a VSCode extension to keep my configuration/settings synchronized across machines. Note that I want VSCode to display on the Windows side but run code and tools from the Linux side (my programming tools won’t be installed on the Windows side). Note that the extension won’t be needed soon since VSCode is going to have settings synchronization built-in.

That said, there are still a bunch of things to coordinate so let me walk you through what My script does and how it does it.

Calling the Windows side from Linux

The first trick is to get the Windows home path since we are going to need it for a few things. This is done using WSL specific linux commands called wslpath and wslvar.

export WINHOME=$(wslpath "$(wslvar USERPROFILE)")

Then we wrap a calling winget from Linux:

# winget calls winget on the windows side
function winget { "cmd.exe /C winget.exe $1 $2 $3";}

Windows package manager

winget is the new Microsoft package manager, the equivalent of homebrew but for Windows and officially developed by Microsoft. As I’m writing this article, winget is still in preview but it’s maturing quickly. Winget must be installed on the Windows side for the script to work, so we are going to need to check if it’s available before going further.

The bash script to check if winget is available on the windows side looks like that:

wingetAlert() {
    echo "winget could not be found"
    echo "make sure winget is installed and available and restart this script - https://www.microsoft.com/en-us/p/app-installer/9nblggh4nns1"
    cmd.exe /C start "https://www.microsoft.com/en-us/p/app-installer/9nblggh4nns1"
    exit
}

# checking that winget is installed on the windows side
ifCMDNotAvail "winget.exe" wingetAlert

We are calling the ifCMDNotAvail function and passing 2 arguments, the name of the executable to check and the function to call if winget isn’t available. The alert prints some text and then opens the Microsoft store page for winget by using the Windows start.exe command.

ifCMDNotAvail is a function using command -v to see if a command available or not, if it’s not, then the second argument is called.

Installing Windows applications

Here is the block of code installing the apps I’d like if they aren’t already installed:

ifPathNotAvailWinInstall "/mnt/c/Program Files/PowerToys/PowerToys.exe" Microsoft.Powertoys
ifCMDNotAvail "wt.exe" Microsoft.WindowsTerminal
ifPathNotAvailWinInstall "$WINHOME/AppData/Local/1Password/app/7/1Password.exe" AgileBits.1Password
ifPathNotAvailWinInstall "/mnt/c/Program Files/VideoLAN/VLC/vlc.exe" VideoLAN.VLC
ifPathNotAvailWinInstall "/mnt/c/Program Files/7-Zip/7z.exe" 7zip
ifPathNotAvailWinInstall "/mnt/c/Program Files (x86)/Adobe/Acrobat Reader DC/Reader/AcroRd32.exe" Adobe.AdobeAcrobatReaderDC

ifCMDNotAvail code installVSCode

Without going through the full explanation, we do a check on the command or path and call winget on the Windows side with the name of the package to install. Pretty straightforward.

Linux setup

Nothing really special here, just installing my tools the linux way locally :) Something interesting to show is how I setup my ssh key.

function setupSSHKey {
    echo "Setting up Git"
    ssh-keygen -t rsa -b 4096 -C "[email protected]"
    echo "new SSH key generated"
    ssh-agent
    ssh-add ~/.ssh/id_rsa
    cat ~/.ssh/id_rsa.pub | clip.exe
    cmd.exe /C start https://github.com/settings/ssh/new
    echo "Your new ssh key was added to your clipboard, add it to GitHub (and consider turning on SSO)"
    echo "Press any key when your key was added to GitHub"
    while true; do
        read -t 3 -n 1
        if [ $? = 0 ] ; then
            break;
        else
            echo "waiting for the keypress"
        fi
    done
}

I’m definitely not a bash expert, far from it but the part that’s interesting here is how I load the ssh key into the windows clipboard and open GitHub so I can paste it right there. I accomplish that by piping the result of the linux cat command into the Windows clip.exe command.

cat ~/.ssh/id_rsa.pub  | clip.exe

Honestly, this blew my mind the first time I wrote this line and it just worked.

So Here it is, the full script. Probably a work in progress, but I’m quite happy with how convenient and maintainable this solution is. I got a new PC and was able to get it up and running in less than 30 minutes 🥳